Ilm library techniques with tivoli storage and ibm total storage products sg247030

1,212 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,212
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Ilm library techniques with tivoli storage and ibm total storage products sg247030

  1. 1. Front coverILM Library: Techniqueswith Tivoli Storage andIBM TotalStorage ProductsLearn about basic ILM conceptsUse TPC for Data to assess ILMreadinessStages to ILMimplementation Charlotte Brooks Giacomo Chiapparini Wim Feyants Pallavi Galgali Vinicius Franco Joseibm.com/redbooks
  2. 2. International Technical Support OrganizationILM Library: Techniques with Tivoli Storage and IBMTotalStorage ProductsFebruary 2006 SG24-7030-00
  3. 3. Note: Before using this information and the product it supports, read the information in “Notices” on page xi.First Edition (February 2006)© Copyright International Business Machines Corporation 2006. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP ScheduleContract with IBM Corp.
  4. 4. Contents Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvPart 1. ILM overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 1. Introduction to ILM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 What is ILM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2 Why ILM is needed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.1 IT challenges and how ILM can help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 ILM elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3.1 Tiered storage management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3.2 Long-term data retention. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.3.3 Data lifecycle management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3.4 Policy-based archive management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.4 Standards and organizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.4.1 Storage Networking Industry Association (SNIA) . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.5 IT Infrastructure Library and value of ILM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.5.1 What is ITIL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.5.2 ITIL management processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.5.3 ITIL and ILM value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Chapter 2. ILM within an On Demand storage environment . . . . . . . . . . . . . . . . . . . . . 21 2.1 Information On Demand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.1.1 Infrastructure Simplification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.1.2 Business Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.1.3 Information Lifecycle Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.2 IBM and ILM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.3 IBM Information On Demand environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.4 Supporting ILM through On Demand storage environment . . . . . . . . . . . . . . . . . . . . . 28 Chapter 3. Implementing ILM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.1 Logical stages in ILM implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.1.1 Assessment and planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.1.2 Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.1.3 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.1.4 Flow diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.2 IBM ILM consulting and services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Chapter 4. Product overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.1 Summary of IBM products for ILM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.2 TotalStorage Productivity Center for Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.2.2 Key aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39© Copyright IBM Corp. 2006. All rights reserved. iii
  5. 5. 4.2.3 Product highlights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 4.3 SAN Volume Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.3.2 Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 4.3.3 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.4 IBM TotalStorage DS family of disk products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.4.1 Enterprise disk storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.4.2 Mid-range disk storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.5 IBM TotalStorage tape solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.5.1 IBM Virtualization Engine TS7510 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.6 Tivoli Storage Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.6.2 Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 4.6.3 Tivoli Storage Manager applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4.6.4 Tivoli Storage Manager APIs and DR550 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.7 DB2 Content Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4.7.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.7.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.7.3 Standards and data model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.8 DB2 CommonStore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.8.1 DB2 CommonStore for Exchange Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.8.2 DB2 CommonStore for Lotus Domino. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.8.3 DB2 CommonStore for SAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.9 More information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61Part 2. Evaluating ILM for your organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Chapter 5. An ILM quick assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 5.1 Initial steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 5.2 Getting business and storage information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 5.3 Defining data collection reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 5.3.1 Creating groups of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 5.3.2 Collecting reports from TPC for Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 5.4 Classifying data and analyzing reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 5.4.1 Types of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.4.2 Data classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.5 Defining actions with classified data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.5.1 Actions for non-business files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.5.2 Actions for duplicate files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.5.3 Actions for temporary files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.5.4 Actions for stale files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.5.5 Actions to RDBMSs space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.6 ILM - Return on investment (ROI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.6.1 Data classification and storage cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 5.6.2 Data management and personnel cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 5.6.3 Long-term retention and non-compliancy penalties cost. . . . . . . . . . . . . . . . . . . . 99 5.6.4 Backup/archiving solutions cost - Disk or tape . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.7 ILM Services offerings from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Chapter 6. The big picture for an ILM implementation framework . . . . . . . . . . . . . . . 103 6.1 The big picture and why you should care about it . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.1.1 Business consulting, assessment, definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.1.2 Application and server hardware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.1.3 Software infrastructure and automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107iv ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  6. 6. 6.1.4 Hardware infrastructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.1.5 Management tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 6.2 What to do now - The many entry points to ILM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115Part 3. Sample solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Chapter 7. ILM initial implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 7.1 Storage management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 7.1.1 Capacity management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7.1.2 Service level management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 7.2 Optimization of storage occupation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 7.2.1 Reclaimable space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 7.2.2 Avoiding over allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 7.3 Tiered storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 7.3.1 What storage devices to use. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 Chapter 8. Enforcing data placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 8.1 Moving from the initial ILM scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 8.2 Requirements for data placement enforcement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 8.2.1 Data classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 8.2.2 Enforcing data placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Chapter 9. Data lifecycle and content management solution . . . . . . . . . . . . . . . . . . . 165 9.1 Moving from the previous steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 9.2 Placement in function of moment in lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 9.2.1 Determining the value of the data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 9.2.2 Placement of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 9.2.3 Movement of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 9.2.4 Using document management systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 9.3 E-mail management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 9.3.1 Reclaim invalid space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 9.3.2 E-mail archiving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 9.4 IBM System Storage Archive Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 9.4.1 Chronological archive retention. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 9.4.2 Event-based retention policy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Contents v
  7. 7. vi ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  8. 8. Figures 1-1 Information Lifecycle Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1-2 Data value changes over time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1-3 ILM elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1-4 Traditional non-tiered storage environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1-5 Multi-tiered storage environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1-6 ILM policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1-7 Information value changes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1-8 Value of information and archive/retrieve management . . . . . . . . . . . . . . . . . . . . . . 15 1-9 SNIA vision for ILM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1-10 ITIL processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2-1 IS, BC, and ILM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2-2 Business Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2-3 Convergence of technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2-4 Information On Demand storage environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2-5 Information Assets and Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3-1 Data classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3-2 Information classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3-3 Storage tiers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3-4 Flow diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4-1 Storage resource management lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4-2 First screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4-3 Availability report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4-4 Asset Report of a computer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4-5 Largest files by computer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 4-6 SVC block virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 4-7 SVC components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4-8 Tivoli Storage Manager components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 4-9 Tivoli Storage Manager for Mail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4-10 Tivoli Storage Manager for Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4-11 Tivoli Storage Manager for Application Servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 4-12 Tivoli Storage Manager for ERP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 4-13 Tivoli Storage Manager for Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 4-14 Tivoli Storage Manager for Space Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 4-15 TSM APIs and DR550 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4-16 Enterprise content management components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4-17 IBM content management portfolio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 5-1 Quick Assessment steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 5-2 Access File Summary report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 5-3 Access Time Summary report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 5-4 Disk Capacity Summary report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 5-5 Oldest Orphaned Files report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 5-6 Storage Access Times report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 5-7 Storage Capacity report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 5-8 Storage Modification Times report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 5-9 Total Freespace report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5-10 User Space Usage report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 5-11 Wasted Space report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 5-12 Largest Files report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81© Copyright IBM Corp. 2006. All rights reserved. vii
  9. 9. 5-13 Duplicate Files report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5-14 File Types Report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5-15 Access Time report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 5-16 Modification Time Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 5-17 Database Storage by Computer report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 5-18 Database Storage by Computer report table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5-19 Total Database Free report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 5-20 Segments with Wasted Space report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 5-21 Access Time Reporting by report group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5-22 Access Time reporting file systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5-23 Modification Time Reporting by report group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 5-24 Modification Time reporting file systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 6-1 ILM implementation framework at service level maturity . . . . . . . . . . . . . . . . . . . . . 105 6-2 Business, assessment, and ongoing tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6-3 Server types and agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 6-4 Software components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6-5 Storage Hardware Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6-6 Placement of WBEM and CIM technology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 7-1 Generic process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7-2 Capacity management tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 7-3 Space usage over time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 7-4 Service level management activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 7-5 Steps to create a service level agreement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 7-6 Overview of space usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 7-7 Top 10 file types using the most space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 7-8 Defining non-business data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 7-9 Ratio between temporary space and used space remains constant . . . . . . . . . . . . 134 7-10 Ratio between temporary used space and total used space increasing . . . . . . . . . 135 7-11 Decrease in ratio between temporary and used space . . . . . . . . . . . . . . . . . . . . . . 136 7-12 Creating a profile - Defining statistics to gather . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 7-13 Defining the file filters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 7-14 Defining the scan systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 7-15 Defining the scan profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 7-16 Generating a report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 7-17 Temporary space report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 7-18 Organizational and project-based file-sharing structure . . . . . . . . . . . . . . . . . . . . . 140 7-19 Moving stale data in a two-tier Tivoli Storage Manager HSM solution. . . . . . . . . . . 142 7-20 TPC for data access time reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 7-21 HSM Data placement in function of time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 7-22 Overview of two-tier HSM implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 7-23 File system unused space reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 7-24 Defining the space allocation trigger level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 7-25 Database unused space report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 7-26 Handling over-allocated file systems and databases. . . . . . . . . . . . . . . . . . . . . . . . 149 7-27 Matching data classes to storage tiers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 8-1 Initial static ILM implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 8-2 Adding automated data placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 8-3 Adding file-based location rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 8-4 Complete picture of data types to tiers mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 8-5 Server tiered volume mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 8-6 Enforcing data placement using TPC for Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 9-1 Adding the lifecycle dimension. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 9-2 The changing value of data over time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167viii ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  10. 10. 9-3 Business process to data mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1689-4 Cost versus benefit for storage placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1709-5 Process example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1739-6 E-mail propagation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1789-7 E-mail archiving diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1799-8 Standard IBM System Storage Archive Manager archive retention. . . . . . . . . . . . . 1819-9 Event driven archiving mechanism - Honoring RETVER. . . . . . . . . . . . . . . . . . . . . 1829-10 Event driven archiving mechanism - honouring RETMIN - case 2 . . . . . . . . . . . . . 183 Figures ix
  11. 11. x ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  12. 12. NoticesThis information was developed for products and services offered in the U.S.A.IBM may not offer the products, services, or features discussed in this document in other countries. Consultyour local IBM representative for information on the products and services currently available in your area. Anyreference to an IBM product, program, or service is not intended to state or imply that only that IBM product,program, or service may be used. Any functionally equivalent product, program, or service that does notinfringe any IBM intellectual property right may be used instead. However, it is the users responsibility toevaluate and verify the operation of any non-IBM product, program, or service.IBM may have patents or pending patent applications covering subject matter described in this document. Thefurnishing of this document does not give you any license to these patents. You can send license inquiries, inwriting, to:IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.The following paragraph does not apply to the United Kingdom or any other country where such provisions areinconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THISPUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer ofexpress or implied warranties in certain transactions, therefore, this statement may not apply to you.This information could include technical inaccuracies or typographical errors. Changes are periodically madeto the information herein; these changes will be incorporated in new editions of the publication. IBM may makeimprovements and/or changes in the product(s) and/or the program(s) described in this publication at any timewithout notice.Any references in this information to non-IBM Web sites are provided for convenience only and do not in anymanner serve as an endorsement of those Web sites. The materials at those Web sites are not part of thematerials for this IBM product and use of those Web sites is at your own risk.IBM may use or distribute any of the information you supply in any way it believes appropriate without incurringany obligation to you.Information concerning non-IBM products was obtained from the suppliers of those products, their publishedannouncements or other publicly available sources. IBM has not tested those products and cannot confirm theaccuracy of performance, compatibility or any other claims related to non-IBM products. Questions on thecapabilities of non-IBM products should be addressed to the suppliers of those products.This information contains examples of data and reports used in daily business operations. To illustrate themas completely as possible, the examples include the names of individuals, companies, brands, and products.All of these names are fictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.COPYRIGHT LICENSE:This information contains sample application programs in source language, which illustrates programmingtechniques on various operating platforms. You may copy, modify, and distribute these sample programs inany form without payment to IBM, for the purposes of developing, using, marketing or distributing applicationprograms conforming to the application programming interface for the operating platform for which the sampleprograms are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, anddistribute these sample programs in any form without payment to IBM for the purposes of developing, using,marketing, or distributing application programs conforming to IBMs application programming interfaces.© Copyright IBM Corp. 2006. All rights reserved. xi
  13. 13. TrademarksThe following terms are trademarks of the International Business Machines Corporation in the United States,other countries, or both: Eserver® DB2 Universal Database™ OS/390® Eserver® DB2® POWER5™ Redbooks (logo) ™ Enterprise Storage Server® Redbooks™ developerWorks® FlashCopy® System Storage™ iSeries™ Informix® Tivoli® xSeries® IBM® TotalStorage® z/OS® Lotus Notes® VideoCharger™ AIX® Lotus® Virtualization Engine™ Domino® Notes® WebSphere®The following terms are trademarks of other companies:JDBC, Streamline, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, othercountries, or both.Excel, Microsoft, Outlook, PowerPoint, Visio, Visual Basic, Windows, and the Windows logo are trademarks of MicrosoftCorporation in the United States, other countries, or both.Intel, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or itssubsidiaries in the United States, other countries, or both.UNIX is a registered trademark of The Open Group in the United States and other countries.Linux is a trademark of Linus Torvalds in the United States, other countries, or both.Other company, product, or service names may be trademarks or service marks of others.xii ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  14. 14. Preface Every organization has large amounts of data to store, use, and manage. For most, this quantity is increasing. However, over time, the value of this data changes. How can we map data to appropriately priced storage media, so that it can be accessed in a timely manner when needed, retained for as long as required, and disposed of when no longer needed? Information Lifecycle Management (ILM) provides solutions. What is ILM? ILM is the process of managing information from creation, through its useful life, to its eventual destruction. In a manner that aligns storage costs with the changing business value of information. We can think of ILM as an integrated solution of five IT management and infrastructure components working together: Service management (service levels), content management, workflow management (or process management), storage management, and storage infrastructure. This IBM® Redbook will help you understand what ILM is, why it is of value to you in your organization, and some suggested ways to implement it using IBM products. It focuses particularly on data life cycle management. Look for other Redbooks™ on topics such as archive and retention management.The team that wrote this redbook This redbook was produced by a team of specialists from around the world working at the International Technical Support Organization, San Jose Center. Charlotte Brooks is an IBM Certified IT Specialist and Project Leader for Storage Solutions at the International Technical Support Organization, San Jose Center. She has 15 years of experience with IBM in storage hardware and software support, deployment, and management. She has written many Redbooks, and has developed and taught IBM classes in all areas of storage and storage management. Before joining the ITSO in 2000, she was the Technical Support Manager for Tivoli® Storage Manager in the Asia Pacific Region. Giacomo Chiapparini is an IBM Certified System Expert for Open Systems Storage Solutions in IBM Global Services Switzerland. He has eight years of practical experience in designing, implementing, and supporting different storage solutions across the country. He is an SNIA certified professional and holds product certifications for Linux®, AIX®, and Windows®. His areas of expertise include storage products, storage networking, and open systems server hardware with corresponding operating systems. Wim Feyants is an IBM Certified IT Specialist in Belgium. He has 11 years of experience in different IT fields. His areas of expertise include storage infrastructure and storage management solution, and designing and implementing them for clients. He has written extensively on different storage-related matters, including a number of Redbooks. Pallavi Galgali is a Software Engineer at the IBM India Software Lab in Pune, India. Pallavi has been involved in development and maintenance projects with products such as SAN File System and Advanced Distributed File System. She has co-authored the IBM Redbook The IBM TotalStorage Solutions Handbook, SG24-5250, and an article on developerWorks® titled A comparison of security subsystems on AIX, Linux, and Solaris. She holds a degree in Computer Engineering from Pune Institute of Computer Technology, India. Her areas of expertise include storage networking, file systems, and device drivers. Vinicius Franco Jose is a Senior IT Specialist at IBM Brazil. He has been in the IT industry for eight years and has extensive experience implementing UNIX® and Storage solutions.© Copyright IBM Corp. 2006. All rights reserved. xiii
  15. 15. His areas of expertise include several TotalStorage® Disk and Tape solutions, Tivoli Storage products, and TPC. He also has experience in storage networking, SAN Volume Controller, and SAN File System. He holds products certifications including AIX, Tivoli Storage Manager, and TPC for Data. He is currently working in IBM Global Services on client support and services delivery. He is also a member of the ILM IT Solution group in Brazil deploying solutions for ILM projects. Figure 1 The team: Wim, Charlotte, Pallavi, Giacomo, Vinicius Thanks to the following people for their contributions to this book: David Bartlett, Larry Heathcote, BJ Klingenberg, Toby Marek, Scott McPeek, Dave Russell, Evan Salop, Chris Saul, Scott Selvig, Alan Stuart, Sergei Varbanov IBM Emma Jacobs, Mary Lovelace, Sangam Racherla International Technical Support Organization, San Jose Center Julie Czubik International Technical Support Organization, Poughkeepsie Center Taya Wyss Enterprise Strategy Group Pillip Mills SNIA Rep for IBMBecome a published author Join us for a two- to six-week residency program! Help write an IBM Redbook dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. Youll team with IBM technical professionals, Business Partners and/or clients.xiv ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  16. 16. Your efforts will help increase product acceptance and client satisfaction. As a bonus, youll develop a network of contacts in IBM development labs, and increase your productivity and marketability. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.htmlComments welcome Your comments are important to us! We want our Redbooks to be as helpful as possible. Send us your comments about this or other Redbooks in one of the following ways: Use the online Contact us review redbook form found at: ibm.com/redbooks Send your comments in an email to: redbook@us.ibm.com Mail your comments to: IBM Corporation, International Technical Support Organization Dept. QXXE Building 80-E2 650 Harry Road San Jose, California 95120-6099 Preface xv
  17. 17. xvi ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  18. 18. Part 1Part 1 ILM overview In this part we introduce basic definitions and concepts for ILM, as well as some of the core IBM and Tivoli products in this solution space.© Copyright IBM Corp. 2006. All rights reserved. 1
  19. 19. 2 ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  20. 20. 1 Chapter 1. Introduction to ILM Information is essential to any business. Organizations have the challenge to efficiently manage information throughout its lifecycle, related to its business value. The quantity of information and its value changes over time, and becomes increasingly costly and complex to store and manage. This chapter discusses the importance of ILM, its benefits, and introduces you to the elements of data lifecycle management.© Copyright IBM Corp. 2006. All rights reserved. 3
  21. 21. 1.1 What is ILM Information Lifecycle Management (ILM) is a process for managing information through its lifecycle, from conception until disposal, in a manner that optimizes storage and access at the lowest cost. ILM is not just hardware or software—it includes processes and policies to manage the information. It is designed upon the recognition that different types of information can have different values at different points in their lifecycle. Predicting storage needs and controlling costs can be especially challenging as the business grows. The overall objectives of managing information with Information Lifecycle Management are to help reduce the total cost of ownership (TCO) and help implement data retention and compliance policies. In order to effectively implement ILM, owners of the data need to determine how information is created, how it ages, how it is modified, and if/when it can safely be deleted. ILM segments data according to value, which can help create an economical balance and sustainable strategy to align storage costs with businesses objectives and information value. The adoption of ILM technologies and processes, as shown in Figure 1-1, turns that strategy into a business reality. Figure 1-1 Information Lifecycle Management1.2 Why ILM is needed In order to run your business efficiently, you need fast access to your stored data. But in today’s business environment, you face increasing challenges: The explosion of the sheer volume of digital information, the increasing cost of storage management, tight regulatory requirements for data retention, and manual business and IT processes that are increasingly complex and error prone. Although the total value of stored information has increased overall, historically, not all data is created equal, and the value of that data to business operations fluctuates over time. This is shown in Figure 1-2 on page 5, and is commonly referred to as the data lifecycle. The existence of the data lifecycle means that all data cannot be treated the same.4 ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  22. 22. Data Base 120 Development Code Email 100 Productivity Files MPEG Data Value 80 60 40 20 0 7 14 21 28 3 6 9 1 5 10 Days Days Days Days Months Months Months Year Years Years Source of graph: Enterprise Strategy Group Time Figure 1-2 Data value changes over time Figure 1-2 shows typical data values of different types of data, mapped over time. Most frequently, the value of data decreases over time, albeit at different rates of decline. However, infrequently accessed or inactive data can become suddenly valuable again as events occur, or as new business initiatives or projects are taken on. Historically, the need to retain information has resulted in a “buy more storage” mentality. However, this approach has only served to increase overall storage management costs and complexity, and has increased the demand for hard-to-find qualified personnel. Executives today are tasked with reducing overall spending while supporting an ever-increasing number of service and application demands. While support and management tasks increase, IT departments are being asked to justify their position by demonstrating business value to the enterprise. IT must also develop and enhance the infrastructure in order to support business initiatives while facing some or all of these data storage issues: Costs associated with e-mail management can reduce employee productivity in many companies. Backup and recovery windows continue to expand as data volumes grow unmanaged. Inactive data consumes valuable, high-performance disk storage space. Duplicate data copies are consuming additional storage space. As data continues to grow and management costs increase, budgets continue to be under pressure.1.2.1 IT challenges and how ILM can help There are many challenges facing business today that make organizations think about managing their information more efficiently and effectively. Among these are some particular issues that might motivate you to develop an ILM strategy and solution: Information and data growing faster than the storage budget. What data can I delete and when? What to keep and for how long? Disk dedicated to specific applications - inhibits sharing. Duplicated copies of files and other data. Where are they and how much space do they use? No mapping of the value of the data to the value of the hardware on which it is stored. Chapter 1. Introduction to ILM 5
  23. 23. Longer time required to backup data, but the window keeps shrinking. Storage performance does not meet requirements. Low utilization of existing assets; for example, in open environments, storage utilization rates of around 30 percent are quite typical. Manual processes causing potential business risk due to errors. Regulatory requirements dictate long-term retention for certain data. Inability to achieve backup/recovery/accessibility objectives for critical data. Inability to grow the support staff to keep up with the demand for storage management in an increasingly complex environment. Multiple backup and restore approaches and processes. Storage management requirements not well defined. In response to these, it is necessary to define specific objectives to support and improve information management: Control demand for storage and create policies for allocation. Reduce hardware, software. and storage personnel costs. Improve personnel efficiency, optimizing system and productivity. Define and enforce policies to manage the lifecycle of data. Define and implement the appropriate storage strategy to address current and future business requirements. In the next section, we describe the major ILM solution components and how they can help you to overcome these challenges, and propose an ILM assessment for planning and design.1.3 ILM elements To manage the data lifecycle and make your business ready for on demand, there are four main elements that can address your business to an ILM structured environment, as shown in Figure 1-3 on page 7. They are: Tiered storage management Long-term data retention Data lifecycle management Policy-based archive management6 ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  24. 24. Tiered Storage Long-Term Data Retention Incorporates tiered storage and Address needs for risk and advanced SAN technologies. compliance objectives; Storage ranging from enterprise disk, midrange leverages Content Management and disk and tape to optimize costs and availability Records Management technologies “The process of managing information, from creation to disposal, in a manner that aligns costs with the changing value of information” Data Life Cycle Management Policy-based Archive Management Exploit Hierarchical Storage Management for E-mail, database and application archive. any data that needs to be protected and Focused offerings driven by efficiency of major retained for a period of time and then disposed. applications Establish policies and automation to move data among different storage systems Figure 1-3 ILM elements In the next four sections we describe each of these elements in detail: 1.3.1, “Tiered storage management” on page 7 1.3.2, “Long-term data retention” on page 9 1.3.3, “Data lifecycle management” on page 12 1.3.4, “Policy-based archive management” on page 141.3.1 Tiered storage management Most organizations today seek a storage solution that can help them manage data more efficiently. They want to reduce the costs of storing large and growing amounts of data and files and maintain business continuity. Through tiered storage, you can reduce overall disk-storage costs, by providing benefits like: Reducing overall disk-storage costs by allocating the most recent and most critical business data to higher performance disk storage, while moving older and less critical business data to lower cost disk storage. Speeding business processes by providing high-performance access to most recent and most frequently accessed data. Reducing administrative tasks and human errors. Older data can be moved to lower cost disk storage automatically and transparently. Typical storage environment Storage environments typically have multiple tiers of data value, such as application data that is needed daily and archive data that is accessed infrequently. But typical storage configurations offer only a single tier of storage, as in Figure 1-4 on page 8, which limits the ability to optimize cost and performance. Chapter 1. Introduction to ILM 7
  25. 25. Figure 1-4 Traditional non-tiered storage environment Multi-tiered storage environment A tiered storage environment is the infrastructure needed to align storage cost with the changing value of information. The tiers will be related to data value. The most critical data is allocated to higher performance disk storage, while less critical business data is allocated to lower cost disk storage. Each storage tier will provide different performance metrics and disaster recovery capabilities. Creating classes and storage device groups is an important step to configure a tiered storage ILM environment. We will provide details of this in later chapters of this book. Figure 1-5 on page 9 shows a multi-tiered storage environment.8 ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  26. 26. Figure 1-5 Multi-tiered storage environment An IBM ILM solution in a tiered storage environment is designed to: Reduce the total cost of ownership (TCO) of managing information. It can help optimize data costs and management, freeing expensive disk storage for the most valuable information. Segment data according to value. This can help create an economical balance and sustainable strategy to align storage costs with business objectives and information value. Help make decisions about moving, retaining, and deleting data, because ILM solutions are closely tied to applications. Manage information and determine how it should be managed based on content, rather than migrating data based on technical specifications. This approach can help result in more responsive management, and offers you the ability to retain or delete information in accordance with business rules. Provide the framework for a comprehensive enterprise content management strategy. Key products of IBM for tiered storage solutions and storage virtualization solutions are: IBM TotalStorage SAN Volume Controller (SVC) IBM TotalStorage DS family of disk storage - DS4x000, DS6000, and DS8000 IBM TotalStorage tape drives, tape libraries, and virtual tape solutions For details of these, see Chapter 4, “Product overview” on page 37.1.3.2 Long-term data retention There is a rapidly growing class of data that is best described by the way in which it is managed rather than the arrangement of its bits. The most important attribute of this kind of data is its retention period, hence it is called retention managed data, and it is typically kept in an archive or a repository. In the past it has been variously known as archive data, fixed Chapter 1. Introduction to ILM 9
  27. 27. content data, reference data, unstructured data, and other terms implying its read-only nature. It is often measured in terabytes and is kept for long periods of time, sometimes forever. In addition to the sheer growth of data, laws and regulations governing the storage and secure retention of business and client information are increasingly becoming part of the business landscape, making data retention a major challenge to any institution. An example of these is the Sarbanes-Oxley Act in the US, of 2002. Businesses must comply with these laws and regulations. Regulated information can include e-mail, instant messages, business transactions, accounting records, contracts, or insurance claims processing, all of which can have different retention periods, for example, for 2 years, for 7 years, or retained forever. Moreover, some data must be kept just long enough and no longer. Indeed, content is an asset when it needs to be kept; however, data kept past its mandated retention period could also become a liability. Furthermore, the retention period can change due to factors such as litigation. All these factors mandate tight coordination and the need for ILM. Not only are there numerous state and governmental regulations that must be met for data storage, but there are also industry-specific and company-specific ones. And of course these regulations are constantly being updated and amended. Organizations need to develop a strategy to ensure that the correct information is kept for the correct period of time, and is readily accessible when it needs to be retrieved at the request of regulators or auditors. It is easy to envisage the exponential growth in data storage that will result from these regulations and the accompanying requirement for a means of managing this data. Overall, the management and control of retention managed data is a significant challenge for the IT industry when taking into account factors such as cost, latency, bandwidth, integration, security, and privacy. Regulations examples It is not within the scope of this book to enumerate and explain the regulations in existence today. For illustration purposes only, we list some of the major regulations and accords in Table 1-1, summarizing their intent and applicability. Table 1-1 Some regulations and accords affecting companies Regulation Intention Applicability SEC/NASD Prevent securities fraud. All financial institutions and companies regulated by the SEC Sarbanes Oxley Act Ensure accountability for public All public companies trading on firms. a U.S. Exchange HIPAA Privacy and accountability for Health care providers and health care providers and insurers, both human and insurers. veterinarian Basel II aka The New Accord Promote greater consistency in Financial industry the way banks and banking regulators approach risk management across national borders. 21 CFR 11 Approval accountability. FDA regulation of pharmaceutical and biotechnology companies10 ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  28. 28. For example, in Table 1-2, we list some requirements found in SEC 17a-4 to which financialinstitutions and broker-dealers must comply. Information produced by these institutions,regarding solicitation and execution of trades and so on, is referred to as compliance data, asubset of retention-managed data.Table 1-2 Some SEC/NASD requirements Requirement Met by Capture all correspondence (unmodified) Capture incoming and outgoing e-mail before [17a-4(f)(3)(v)]. reaching users. Store in non-rewritable, non-erasable format Write Once Read Many (WORM) storage of all [17a-4(f)(2)(ii)(A)]. e-mail, all documents. Verify automatically recording integrity and Validated storage to magnetic, WORM. accuracy [17a-4(f)(2)(ii)(B)]. Duplicate data and index storage Mirrored or duplicate storage servers (copy [17a-4(f)(3)(iii)]. pools). Enforce retention periods on all stored data and Structured records management. indexes [17a-4(f)(3)(iv)(c)]. Search/retrieve all stored data and indexes High-performance search retrieval. [17a-4(f)(2)(ii)(D)].IBM ILM data retention strategyRegulations and other business imperatives, as we just briefly discussed, stress the need foran Information Lifecycle Management process and tools to be in place. The uniqueexperience of IBM with the broad range of ILM technologies, and its broad portfolio ofofferings and solutions, can help businesses address this particular need and provide themwith the best solutions to manage their information throughout its lifecycle. IBM provides acomprehensive and open set of solutions to help.IBM has products that provide content management, data retention management, andsophisticated storage management, along with the storage systems to house the data. Tospecifically help companies with their risk and compliance efforts, the IBM Risk andCompliance framework is another tool designed to illustrate the infrastructure capabilitiesneeded to help address the myriad of compliance requirements. Using the framework,organizations can standardize the use of common technologies to design and deploy acompliance architecture that may help them deal more effectively with compliance initiatives.For more details about the IBM Risk and Compliance framework, visit:http://www-306.ibm.com/software/info/openenvironment/rcf/Key products of IBM for data retention and compliance solutions are: IBM Tivoli Storage Manager, including IBM System Storage™ Archive Manager IBM DB2® Content Manager Family, which includes DB2 Content Manager, Content Manager OnDemand, CommonStore for Exchange Server, CommonStore for Lotus® Domino®, and CommonStore for SAP IBM DB2 Records Manager IBM TotalStorage DS4000 with S-ATA disks IBM System Storage DR550 IBM TotalStorage Tape (including WORM) products Chapter 1. Introduction to ILM 11
  29. 29. For details on these products, see Chapter 4, “Product overview” on page 37. Important: The IBM offerings are intended to help clients address the numerous and complex issues relating to data retention in regulated and non-regulated business environments. Nevertheless, each client’s situation is unique, and laws, regulations, and business considerations impacting data retention policies and practices are constantly evolving. Clients remain responsible for ensuring that their information technology systems and data retention practices comply with applicable laws and regulations, and IBM encourages clients to seek appropriate legal counsel to ensure their compliance with those requirements. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the client is in compliance with any law.1.3.3 Data lifecycle management At its core, the process of ILM moves data up and down a path of tiered storage resources, including high-performance, high-capacity disk arrays, lower-cost disk arrays such as serial ATA (SATA), tape libraries, and permanent archival media where appropriate. Yet ILM involves more than just data movement; it encompasses scheduled deletion and regulatory compliance as well. Because decisions about moving, retaining, and deleting data are closely tied to application use of data, ILM solutions are usually closely tied to applications. ILM has the potential to provide the framework for a comprehensive information-management strategy, and helps ensure that information is stored on the most cost-effective media. This helps enable administrators to make use of tiered and virtual storage, as well as process automation. By migrating unused data off of more costly, high-performance disks, ILM is designed to help: Reduce costs to manage and retain data. Improve application performance. Reduce backup windows and ease system upgrades. Streamline™ data management. Allow the enterprise to respond to demand—in real-time. Support a sustainable storage management strategy. Scale as the business grows. ILM is designed to recognize that different types of information can have different value at different points in their lifecycle. As shown in Figure 1-6 on page 13, data can be allocated to a specific storage level aligned to its cost, with policies defining when and where data will be moved.12 ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  30. 30. Figure 1-6 ILM policiesBut, sometimes, the value of a piece of information may change and data that was previouslyinactive and was migrated to a lower-cost storage now could be needed and should beprocessed in a high-performance disk. A data lifecycle management policy can be defined tomove the information back to enterprise storage, making the storage cost aligned to datavalue, as illustrated in Figure 1-7.Figure 1-7 Information value changesKey products of IBM for lifecycle management are: IBM TotalStorage Productivity Center IBM TotalStorage SAN Volume Controller (SVC) IBM Tivoli Storage Manager including IBM System Storage Archive Manager IBM Tivoli Storage Manager for Space Management Chapter 1. Introduction to ILM 13
  31. 31. For details of these products, see Chapter 4, “Product overview” on page 37.1.3.4 Policy-based archive management As businesses of all sizes migrate to e-business solutions and a new way of doing business, they already have mountains of data and content that have been captured, stored, and distributed across the enterprise. This wealth of information provides a unique opportunity. By incorporating these assets into e-business solutions, and at the same time delivering newly generated information media to their employees and clients, a business can reduce costs and information redundancy and leverage the potential profit-making aspects of their information assets. Growth of information in corporate databases such as Enterprise Resource Planning (ERP) systems and e-mail systems makes organizations think about moving unused data off the high-cost disks. They need to: Identify database data that is no longer being regularly accessed and move it to an archive where it remains available. Define and manage what to archive, when to archive, and how to archive from the mail system or database system to the back-end archive management system. Database archive solutions can help improve performance for online databases, reduce backup times, and improve application upgrade times. E-mail archiving solutions are designed to reduce the size of corporate e-mail systems by moving e-mail attachments and/or messages to an archive from which they can easily be recovered if needed. This action helps reduce the need for end-user management of e-mail, improves the performance of e-mail systems, and supports the retention and deletion of e-mail. The way to do this is to migrate and store all information assets into an e-business enabled content manager. ERP databases and e-mail solutions generate large volumes of information and data objects that can be stored in content management archives. An archive solution allows you to free system resources, while maintaining access to the stored objects for later reference. Allowing it to manage and migrate data objects gives a solution the ability to have ready access to newly created information that carries a higher value, while at the same time still being able to retrieve data that has been archived on less expensive media, as shown in Figure 1-8 on page 15.14 ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  32. 32. Figure 1-8 Value of information and archive/retrieve management Key products of IBM for archive management are: IBM Tivoli Storage Manager including IBM System Storage Archive Manager IBM DB2 Content Manager family of products IBM DB2 CommonStore family of products For details o these products, see Chapter 4, “Product overview” on page 37.1.4 Standards and organizations The success and adoption of any new technology, and any improvement to existing technology, is greatly influenced by standards. Standards are the basis for the interoperability of hardware and software from different, and often rival, vendors. Although standards bodies and organizations such as the Internet Engineering Task Force (IETF), American National Standards Institute (ANSI), and International Organization for Standardization (ISO) publish these formal standards, other organizations and industry associations, such as the Storage Networking Industry Association (SNIA), play a significant role in defining the standards and market development and direction.1.4.1 Storage Networking Industry Association (SNIA) The Storage Networking Industry Association is an international computer system industry forum of developers, integrators, and IT professionals who evolve and promote storage networking technology and solutions. SNIA was formed to ensure that storage networks become efficient, complete, and trusted solutions across the IT community. IBM is one of the founding members of this organization. SNIA is uniquely committed to networking solutions into a broader market. SNIA is using its Storage Management Initiative (SMI) and its Storage Management Initiative Specification (SMI-S) to create and promote adoption of a highly Chapter 1. Introduction to ILM 15
  33. 33. functional interoperable management interface for multi-vendor storage networking products. SMI-S makes multi-vendor storage networks simpler to implement and easier to manage. IBM has led the industry in not only supporting the SMI-S initiative, but also using it across its hardware and software product lines. The specification covers fundamental operations of communications between management console clients and devices, auto-discovery, access, security, the ability to provision volumes and disk resources, LUN mapping and masking, and other management operations. Data Management Forum SNIA has formed the Data Management Forum (DMF) to focus on defining, implementing, qualifying, and teaching improved methods for the protection, retention, and lifecycle management of data. Vision for ILM by SNIA and DMF The Data Management Forum defines ILM as a new management practice for the datacenter. ILM is not a specific product, nor is it just about storage and data movement to low-cost disk. It is a standards-based approach to automating datacenter operations by using business requirements, business processes, and the value of information to set policies and service level objectives for how the supporting storage, compute, and network infrastructure operates. The key question that flows from this vision of ILM is How do we get there?, since these capabilities do not fully exist today. This is the work of SNIA and the Data Management Forum: To unify the industry towards a common goal, to develop the relevant standards, to facilitate interoperability, and to conduct market education around ILM. Figure 1-9 illustrates the SNIA vision for ILM. Figure 1-9 SNIA vision for ILM For additional information about the various activities of SNIA and DMF see its Web site at: http://www.snia.org1.5 IT Infrastructure Library and value of ILM The intent of this section is introduce you to the IT Infrastructure Library (ITIL) and the value of ILM within the ITIL methodology. We begin by defining ITIL and its Service Support processes.16 ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  34. 34. 1.5.1 What is ITIL ITIL is a process-based methodology used by IT departments to verify that they can deliver IT services to end users in a controlled and disciplined way. It incorporates a set of best practices that are applicable to all IT organizations, no matter what size or what technology is used. ITIL is used to create and deliver service management processes. These tasks are made easier by the use of service and system management tools. Over recent decades, multiple IT process models have been developed. ITIL is the only one that is not proprietary. Late 1970s: Information Systems Management Architecture (ISMA) (IBM) Late 1980s: IT Infrastructure Library V1 (ITIL) (CCTA - now OGC) 1995: IT Process Model (ITPM) (IBM) 2000: Enterprise Operational Process Framework (IBM) 2000: IT Service Management Reference Model (ITSM) (HP) 2000–2001: Microsoft® Operations Framework (MOF) (Microsoft) 2001–2002: IT Infrastructure Library V2 (ITIL) (OGC) Note: ITIL is a registered trademark of the OGC. OGC is the UK Governments Office of Government Commerce. CCTA is the Central Computer and Telecommunications Agency. ITIL has a library of books describing best practices for IT services management that describe goals, activities, and inputs and outputs of processes. It is a set of best practices. ITIL has a worldwide approach to IT management and its methodology sets that specific procedures can vary from organization to organization. ITIL is not tied to any particular vendor, and IBM has been involved with ITIL since its inception in 1988.1.5.2 ITIL management processes The ITIL approach to creating and managing service management processes is widely recognized around the world and the adoption of its principles is clearly growing, as evidenced by new groups appearing in more countries every year. The service management disciplines are grouped into the two areas of Service Support and Service Delivery. There are now eleven basic processes used in the areas of Service Support and Service Delivery, as shown in Figure 1-10 on page 18. Since it can take a long time to implement these disciplines it is not uncommon to find only some of the processes in use initially. Chapter 1. Introduction to ILM 17
  35. 35. Core ITIL Service Management Processes Capacity Management Availability IT Business Management Continuity Provide quality, Service Delivery cost-effective IT services Financial Service Level Management Management Release Configuration Management Management Provide stability and Service Support Change Service flexibility for Management Desk IT service Incident & provision Problem Management Figure 1-10 ITIL processes Now we briefly explain each component of Service Support and Service Delivery. Service Support The processes in the Service Support group are all concerned with providing stability and flexibility for the provisioning of IT Services. Configuration Management Configuration Management is responsible for registering all components in the IT service (including clients, contracts, SLAs, hardware and software components, and more) and maintain a repository of configurable attributes and relationships between the components. Service Desk The Service Desk acts as the main point-of-contact for the users of the service. Incident Management Incident Management registers incidents, allocates severity, and coordinates the efforts of the support teams to ensure timely and correct resolution of problems. Escalation times are noted in the SLA and are as such agreed between the client and the IT department. Incident Management also provides statistics to Service Level Management to demonstrate the service levels achieved. Problem Management Problem Management implements and uses procedures to perform problem diagnosis and identify solutions that correct problems. It registers solutions in the configuration repository, and agrees on escalation times internally with Service Level Management during the SLA negotiation. It provides problem resolution statistics to support Service Level Management.18 ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  36. 36. Change ManagementChange Management ensures that the impact of a change to any component of a service iswell known, and the implications regarding service level achievements are minimized. Thisincludes changes to the SLA documents and the Service Catalog, as well as organizationalchanges and changes to hardware and software components.Release ManagementRelease Management manages the master software repository and deploys softwarecomponents of services. It deploys changes at the request of Change Management, andprovides management reports on the deployment.Service DeliveryThe processes in the Service Delivery group are all concerned with providing quality,cost-effective IT services.Service Level ManagementThe purpose of Service Level Management is to manage client expectations and negotiateService Delivery Agreements. This involves finding out the client requirements anddetermining how these can best be met within the agreed budget. Service Level Managementworks together will all IT disciplines and departments to plan and ensure delivery of services.This involves setting measurable performance targets, monitoring performance, and takingaction when targets are not met.Financial Management for IT ServicesFinancial Management registers and maintains cost accounts related to the usage of ITservices. It delivers cost statistics and reports to Service Level Management to assist inobtaining the right balance between service cost and delivery. It assists in pricing the servicesin the Service Catalogue and Service Level Agreements.IT Service Continuity ManagementService Continuity Management plans and ensures the continuing delivery—or minimumoutage—of the service by reducing the impact of disasters, emergencies, and majorincidents. This work is done in close collaboration with the company’s business continuitymanagement, which is responsible for protecting all aspects of the company’sbusiness—including IT.Capacity ManagementCapacity Management is responsible for planning and ensuring that adequate capacity withthe expected performance characteristics is available to support the Service Delivery. Itdelivers capacity usage, performance, and workload management statistics, as well as trendanalysis to Service Level Management.Availability ManagementAvailability Management is responsible for planning and ensuring the overall availability of theservices. It provides management information in the form of availability statistics—includingsecurity violations—to Service Level Management. This discipline may also includenegotiating underpinning contracts with external suppliers, and a definition of maintenancewindows and recovery times. Chapter 1. Introduction to ILM 19
  37. 37. 1.5.3 ITIL and ILM value ILM is a service-based solution with policies and processes. The ITIL methodology has the processes needed for delivery and support storage services to manage the lifecycle of information. The ILM components tiered-storage, archive management, long-term retention, and data lifecycle management, aligned to ITIL processes, are a powerful solution for IT organizations to manage their data. By implementing ILM within the ITIL methodology, they will be able to achieve its objectives—enabling the management of data lifecycle, and providing quality, stability, flexibility, and cost-effective IT services.20 ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products
  38. 38. 2 Chapter 2. ILM within an On Demand storage environment This chapter explains how ILM fits in to the strategy for IBM On Demand Business. It briefs on the pillars of Information On Demand, that is: Infrastructure Simplification Business Continuity Information Lifecycle Management It talks about the IBM On Demand storage environment, which helps organizations achieve Information On Demand and discusses ILM components of this environment.© Copyright IBM Corp. 2006. All rights reserved. 21
  39. 39. 2.1 Information On Demand In today’s ever more competitive and growing business environment, information is an increasingly valuable, but costly organizational asset. The volume of information is growing very rapidly in most organizations. And with this, the need to protect and manage information also continues to increase. Organizations are seeking to reduce costs, improve efficiency, and increase effectiveness by aligning IT investments according to information value and business needs. This is a step towards Information On Demand. With Information On Demand, business can respond with flexibility and speed to client requirements and market opportunity. Getting there involves three aspects: Infrastructure Simplification (IS): Simplification of the underlying IT infrastructure and its management to lower the cost and complexity Business Continuity (BC): Assuring security and durability of information Information Lifecycle Management (ILM): Efficiently managing information over its lifecycle Figure 2-1 IS, BC, and ILM2.1.1 Infrastructure Simplification Infrastructure simplification is a process by which organizations contain expenses, enable business growth, and reduce operational risks by optimizing IT resources. Simplified infrastructures hold the promise of improved system optimization and Total Cost of Ownership (TCO), higher personnel productivity, and greater application availability through infrastructure resiliency. IBM products are designed to help clients obtain these benefits through consolidation, virtualization, and automated management. Once simplified, the infrastructure can be better managed for lower cost and with fewer errors. Note: For more information about Infrastructure Simplification, see: http://www-1.ibm.com/servers/storage/solutions/is/2.1.2 Business Continuity The business climate in todays on demand era is highly competitive. Clients, employees, suppliers, and business partners expect to be able to tap into your information at any hour of the day from any corner of the globe. If you have continuous business operations, then22 ILM Library: Techniques with Tivoli Storage and IBM TotalStorage Products

×