© 2013 IBM CorporationIBM InfoSphere Optim Technical OverviewTom Love – Optim Senior IT Client Technical ProfessionalApril...
© 2012 IBM Corporation2 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformatio...
© 2012 IBM Corporation3 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformatio...
© 2012 IBM Corporation4 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformatio...
© 2012 IBM Corporation5 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformatio...
© 2012 IBM Corporation6 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformatio...
© 2012 IBM Corporation7 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformatio...
© 2012 IBM Corporation8 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformatio...
© 2012 IBM Corporation9 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformatio...
© 2012 IBM Corporation10 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation11 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation12 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation13 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation14 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation15 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation16 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2013 IBM CorporationIBM Optim TDM
© 2012 IBM Corporation18 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation19 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2013 IBM CorporationIBM Optim Data Privacy
© 2012 IBM Corporation21 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation22 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation23 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation24 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation25 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation26 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation27 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation28 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation29 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2013 IBM CorporationOptim ArchiveManaging Data Growth Retirement and Compliance .
© 2012 IBM Corporation31 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation32 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation33 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation34 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation35 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation36 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation37 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation38 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2013 IBM CorporationOptim ArchivePortfolio Optimization/Decommissioning
© 2012 IBM Corporation40 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation41 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation42 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation43 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation44 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation45 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation46 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation47 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation48 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation49 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation50 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation51 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation52 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation53 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation54 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation55 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation56 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation57 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
© 2012 IBM Corporation58 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformati...
Upcoming SlideShare
Loading in …5
×

Using Optim and IMS - IMS UG April 2013 Bloomington

697 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
697
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
44
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Using Optim and IMS - IMS UG April 2013 Bloomington

  1. 1. © 2013 IBM CorporationIBM InfoSphere Optim Technical OverviewTom Love – Optim Senior IT Client Technical ProfessionalApril 2013
  2. 2. © 2012 IBM Corporation2 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementValue PropositionEnterprise Data ManagementProduction Databases Test and Development Databases• Segregate Data & Move toArchive• Deploy Tiered StorageStrategies• Retain Data According toValue• Simplify InfrastructureManageApplicationData Growth• Decommission Redundant orObsolete Apps• Gain Control of ApplicationPortfolio• Retain Access to LegacyData• Retire Apps and RepurposeIT Assets• Migrate Apps from High toLow Cost Platforms• Preserve Historical DataEnablePortfolioOptimization• Protect PII Data• Apply Single DataMasking Solution• Use Range of MaskingTechniques• Maintain ReferentialIntegrity• Maintain ContextualLook and FeelEnsureDataPrivacy• Right size Test Apps• Repeatable Process• Quickly Deploy New Apps• Future proof AppsSpeedApplicationDeployment
  3. 3. © 2012 IBM Corporation3 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementInformation Management3Supporting Enterprise EnvironmentsSingle, scalable, interoperable data management solution provides a central point to deploy policiesto extract, store, port, and protect application data records from creation to deletionData Growth Application Retirement Data Privacy Test Data ManagementInfoSphere DiscoveryOPTIM™DATACOM
  4. 4. © 2012 IBM Corporation4 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementCheck TestResultsCreate/ ModifyApplicationCreate TestEnvironmentInspect and AddData for TestingRefresh Test DataMove toProductionTestFail PassA Typical Software Development LifecycleInformation ManagementProduct Overview : Optim Test Database ManagementSubsettingSubsettingDiscovery
  5. 5. © 2012 IBM Corporation5 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementOptim™ Data Privacy SolutionProductionContextual,Application- Aware,Persistent DataMaskingContextual,Application- Aware,Persistent DataMaskingEBS / OracleCustom /SybaseSiebel / DB2TestEBS / OracleCustom /SybaseSiebel / DB2Substitute confidential information with fictionalized dataDeploy multiple masking algorithmsProvide consistency across environments and iterationsEnable off-shore testingProtect private data in non-production environments
  6. 6. © 2012 IBM Corporation6 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementCurrentProductionHistoricalSelective RestoreRestoredArchivesReportingDataHistoricalDataReferenceDataArchiveComplete Business Object provides historical reference snapshot of business activityStorage device independence enables ILMImmutable file format enables data retention complianceIBM® Optim™ Data Growth Solution: ArchivingUniversal Access to Application DataApplication Application XML ODBC / JDBCIBM MashUPIBM MashUPUniversal Access to Application DataApplication XML ODBC / JDBC
  7. 7. © 2012 IBM Corporation7 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementOptim z/OS Implementation
  8. 8. © 2012 IBM Corporation8 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementOptim Directory– Supplements information stored in the database– Maintains product definitions and tracks processing– Stores database connection information (DB Aliases)– Stores user-defined relationshipsRelationshipsOptimDIRECTORYTablesStored in Database- Catalog- System Tables- Data DictionaryReferentialIntegrityRulesAccessDefinitionsDB AliasesMapsThe Optim Directory
  9. 9. © 2012 IBM Corporation9 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementDatabase AliasEstablishing the Database ConnectionOptim view of a database connection– High-level qualifier for database object names• DBalias.creatorid.objectname– Enables cross-Database access– Saved in Optim DirectoryRelationshipsDB AliasesColumn MapsTable MapsACCTS DBCUSTINFO DBSALES DBOptimDIRECTORYTables
  10. 10. © 2012 IBM Corporation10 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementOPTIM Concepts: ArchitectureStorageProfileDB Source/DestinationServer Name• Server address or name•DB Alias• Connectivity via DB Client softwareWork Directory• Server File SystemStorage Profile• Storage and retention policyServer Name• Server address or name•DB Alias• Connectivity via DB Client softwareWork Directory• Server File SystemStorage Profile• Storage and retention policyTemporaryData…ServerNameWorkDirectoryArchiveArchitectureOffline Storage.( Tivoli TSM )Enterprise Storage( IBM DR550 )Document Storage( Tivoli )Optim Directory
  11. 11. © 2012 IBM Corporation11 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementOPTIM Concepts: Architecture with Extended Data SourcesDB Source/DestinationServerNameArchitectureTeradata DatabaseServerz/OS IMS or VSAMDatacom DBIDMSAny other Structured ddatasourceIBM Federation ServerOptim Directory
  12. 12. © 2012 IBM Corporation12 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementDB Relationships are automatically derived from database RI rulesApplication Specified Relationships• Can be defined individually to Optim• Can be imported into Optim from DDLShared by all Optim componentsOPTIMRelationshipsOptimDIRECTORYTablesReferentialIntegrityRulesAccessDefinitionsDB AliasesMapsStored in Database- Catalog- System Tables- Data DictionaryA Word About Relationships...
  13. 13. © 2012 IBM Corporation13 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementExtended RelationshipsSE00582SE012SC00234SC005NW00564NW012NC00123(NC)003MANAGER_IDChar (7)SALESMAN_IDChar (5)Sales Table District TableExample 1Using Substr FunctionParent Table SalesSubstr(SALESMAN_ID,1,2)Child Table DistrictDISTRICT_CD00582NW00234SE00564SC00123NCMANAGER_NOChar (5)DISTRICT_CDChar (2)Example 2Using Concat FunctionParent Table SalesMANAGER_IDChild Table DistrictDISTRICT_CD || MANAGER_NO
  14. 14. © 2012 IBM Corporation14 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementExtended RelationshipsM35M18F56F45SEXChar (1)AGEInteger250145135132RateNumeric (5,0)AgeIntegerSales Table Female_Rates TableParent Table Child TableSales Male_RatesSex “M”Age AgeSales Female_RatesSex “F”Age Age 250145135318RateNumeric (5,0)AgeIntegerMale_Rates TableExample 3Data Driven Relationships
  15. 15. © 2012 IBM Corporation15 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementOur Unique Capability: Complete Business ObjectExample:General Ledger
  16. 16. © 2012 IBM Corporation16 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementExtract - Federated Data SupportRetek / Oracle Other apps / anyDBMSCustom InventoryManagement / DB2Complete BusinessObject CapturesEnd to EndBusiness Process
  17. 17. © 2013 IBM CorporationIBM Optim TDM
  18. 18. © 2012 IBM Corporation18 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementOPTIM TDM Relational Extract FacilityCreating and maintaining test data basesMigrating dataThe data and/or the object metadata can be extractedExtractFile-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----CUSTOMERSORDERSDETAILS-- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLTESTDB-- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLQADBPRODDBLOADEXTRACTINSERT/UPDATELoadFilesCreate -- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLNew_DBADTMCM
  19. 19. © 2012 IBM Corporation19 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementOptim’s Relational Compare FacilitySingle-table or multi-table compareCreates compare file and/or compare Report of resultsFor application testing, QA, and to verify database contentsEnhances productivity by finding unexpected changes in the dataSOURCE 1SOURCE 2COMPAREPROCESSOptimCOMPAREFILEOptimCompareREPORT........................................................................................................................Interactive BrowseVerify TestResults
  20. 20. © 2013 IBM CorporationIBM Optim Data Privacy
  21. 21. © 2012 IBM Corporation21 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementDuring Extract ProcessOrStandalone Convert ProcessOrDuring Insert/Load ProcessTransform or Replace sensitive data usingStandard mapping rules: Literals, Special Registers,Expressions, Default Values,Look-up tablesComplex mapping rules: User exitsDe-Identify test dataProductionDataExtractandConvertMaskedTestData
  22. 22. © 2012 IBM Corporation22 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementExtractFile-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----Transform / masksensitive data-- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLTESTDB-- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLQADBLOADINSERT/UPDATELoadFilesData Privacy in Application TestingExtract a relationally intact subsetfrom production database(s)• Extract data and/or object definitions• Define a new set of test tables• Apply masking during population process• Extract file may be reused but contains un-Masked data• Good process for testing masks-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ----CUSTOMERSORDERSDETAILS-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----CUSTOMERSORDERSDETAILS-- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLNewDBCreateLegacyFilesBKORDERSelectionCriteria
  23. 23. © 2012 IBM Corporation23 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementExtractFile-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLTESTDB-- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLQADBLOADINSERT/UPDATELoadFilesTransform / masksensitive dataData Privacy in Application TestingExtract a relationally intact subsetfrom production database(s)• Extract data and/or object definitions in pre-masked file• Use pre-masked Extract file to create new set of tables• Convert Pre-masked extract file data into second masked extract file•Share masked extract file to be reused for population step• Good practice for testing masks using COMPARE-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ----CUSTOMERSORDERSDETAILS-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----CUSTOMERSORDERSDETAILSMaskedExtractFile-- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLNewDBCreateLegacyFilesBKORDERSelectionCriteria
  24. 24. © 2012 IBM Corporation24 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementExtractFile-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----Transform / masksensitive data-- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLTESTDB-- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLQADBLOADINSERT/UPDATELoadFilesData Privacy in Application TestingExtract a relationally intact subsetfrom production database(s)• Most Secure Approach• Extract data only• Convert during extract•Extract file already contains masked data•Can be shared with testers to reuse-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ----CUSTOMERSORDERSDETAILS-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----CUSTOMERSORDERSDETAILSLegacyFilesBKORDERSelectionCriteria
  25. 25. © 2012 IBM Corporation25 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementExtractFile-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----Transform / masksensitive data-- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLTESTDB-- ---- ---- ---- ------- ----CUST-- ---- ---- ---- ------- ----ORD-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETLQADBLOADINSERT/UPDATELoadFilesData Privacy in Application TestingExtract a relationally intact subsetfrom production database(s)• Most Secure Approach• Extract data only• Convert during extract•Extract file already contains masked data•Can be shared with testers to reuse-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ----CUSTOMERSORDERSDETAILS-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----CUSTOMERSORDERSDETAILSOnly Users authorized to see Private data
  26. 26. © 2012 IBM Corporation26 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementConsistentMaskingacross theEnterpriseMasked fieldsare consistentData is masked132009824157342266SS#s132009824157342266SS#sDB2323457245134235489SSN#s323457245134235489SSN#sClient Billing Application
  27. 27. © 2012 IBM Corporation27 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementExample 2Example 2Example 1Example 1PersNbr FstNEvtOwn LstNEvtOwn27645 Elliot Flynn27645 Elliot FlynnEvent TableEvent TablePersNbr FstNEvtOwn LstNEvtOwn10002 Pablo Picasso10002 Pablo PicassoEvent TableEvent TablePersonal Info TablePersonal Info TablePersNbr FirstName LastName08054 Alice Bennett19101 Carl Davis27645 Elliot FlynnPersonal Info TablePersonal Info TablePersNbr FirstName LastName10000 Jeanne Renoir10001 Claude Monet10002 Pablo PicassoReferential integrity is maintained with keypropagationPatient InformationPatient InformationPatient InformationPatient No. SSNNameAddressCity State ZipPatient No. SSNNameAddressCity State Zip112233 123-45-6789Amanda Winters40 Bayberry DriveElgin IL 60123123456 333-22-4444Erica Schafer12 Murray CourtAustin TX 78704Data is masked with contextually correctdata to preserve integrity of test dataData Privacy – Key Propagation
  28. 28. © 2012 IBM Corporation28 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementCust ID Item # Order Date27645 80-2382 20 June 200427645 86-4538 10 October 2005Customers TableOrders TableCust ID Name Street08054 Alice Bennett 2 Park Blvd19101 Carl Davis 258 Main27645 Elliot Flynn 96 AvenueOriginal DataCust ID Item # Order Date27645 80-2382 20 June 200427645 86-4538 10 October 2005Customers TableOrders TableCust ID Name Street10000 Auguste Renoir Mars2310001 Claude Monet Venus2410002 Pablo Picasso Saturn25Without PropagationNow theseare Orphans!Why Propagation is Important
  29. 29. © 2012 IBM Corporation29 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementMasking FunctionsColumn MapMap unlike column namesTransform/mask sensitive dataDatatype conversionsColumn-level semantic date agingLiteralsRegistersCalculationsDefault valuesSubstringExitsCurrency conversionSocial SecurityCredit CardEmailHash LookupLookupRandom LookupNAME tablesADDRESS tableShuffleString manipulationPROP……
  30. 30. © 2013 IBM CorporationOptim ArchiveManaging Data Growth Retirement and Compliance .
  31. 31. © 2012 IBM Corporation31 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementCurrentProductionHistoricalSelective RestoreRestoredArchivesReportingDataHistoricalDataReferenceDataArchiveComplete Business Object provides historical reference snapshot of business activityStorage device independence enables ILMImmutable file format enables data retention complianceIBM® Optim™ Data Growth Solution: ArchivingUniversal Access to Application DataApplication Application XML ODBC / JDBCIBM MashUPUniversal Access to Application DataApplication XML ODBC / JDBC
  32. 32. © 2012 IBM Corporation32 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementCurrent Data0-2 yrs.ProductionDatabaseInformation LifecycleOffline Archive7+ yrs.Off-line Retention PlatformCD,Tape,Optical, WORMHP StorageWorks™,NetApp NearStore® SnapLock™,IBM Total Storage® solutions(including the DR550)EMC Centera™.Archive File ManagementCompressedArchive FileArchiveReporting/Query ToolsusingODMCognos, Discoverer,SQLPLUS, Etc.Non DBMSRetention PlatformATA File ServerCenteraDR550Etc.On/Near-Line Archive5-6 yrs.Active Historical On-Line3-4 yrs.RestoreArchiveDatabaseNative AccessOptional
  33. 33. © 2012 IBM Corporation33 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementDefine the data to be deletedArchive all dataDelete orders and details after they are safelyarchivedPreserve semantic intelligenceCUSTOMERSORDERSDETAILSRETAINDELETE
  34. 34. © 2012 IBM Corporation34 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementCreate the archive-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----CUSTOMERS-- -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ----ORDERS-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETAILSProductionDatabaseArchiveFileDELETEOptim Directory( Tables)
  35. 35. © 2012 IBM Corporation35 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementApplications accessing the Archive FilesOptim DirectoryTablesArchiveLibraryRESEARCH/BROWSEUse the OPTIM Archive ODM OptionDirect Access within Your Application using standard SQLDefines data-sources for any ODBC or JDBC applicationJoins between multiple data-sourcesarchive files and database tablesEnd-UserQuery / ReportingODM
  36. 36. © 2012 IBM Corporation36 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementArchive CollectionsCollection AArchive 1,2Collection AArchive 1,2Collection BArchive 1,2,3,4Collection BArchive 1,2,3,4Archive 1Archive 1CustomerCustomerCustomerCustomerOrder(Y01)Order(Y01)Detail(Y01)Detail(Y01)CustomerCustomerOrder(Y01, Y02)Order(Y01, Y02)Detail(Y01, Y02)Detail(Y01, Y02)Credit CardCredit CardODBC/JDBCODBC/JDBCArchive 2Archive 2Order(Y2001)Order(Y2001)Detail(Y2001)Detail(Y2001)Archive 3Archive 3Order(Y2002)Order(Y2002)Detail(Y2002)Detail(Y2002)Archive 4Archive 4Credit CardCredit CardPhysicalLogicalTech OverviewTech Overview
  37. 37. © 2012 IBM Corporation37 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementArchive ArchiveArchive ArchiveIndex IndexIndex Index• Indexes are used to quicklylocate specific columnvalues in an archive file.• Indexes greatly increasethe performance whensearching large archives• Indexes provide quickaccess to data during anediscovery, audit or othertypes of request• Without indexes access toarchive data is extremelyslow because the entirefile(s), which can be verylarge, need to be searchedfor the specific data that isrequired.Archive Collections= Virtual DatabaseArchive Indexes Tech OverviewTech Overview
  38. 38. © 2012 IBM Corporation38 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementRestoring Archived DataRepositoryArchiveFilesRESEARCH/BROWSEProduction/StagingDatabaseMetadataMappingRESTORE-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----CUSTOMERS-- -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ----ORDERS-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----DETAILSDatato Restore
  39. 39. © 2013 IBM CorporationOptim ArchivePortfolio Optimization/Decommissioning
  40. 40. © 2012 IBM Corporation40 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementPortfolio Optimization/DecommissioningRemove a system from active service but…Retain access to its business-critical data– Retire an application– Consolidate redundant systems into single “enterprisestandard”– Migrate portfolio to lower-cost platform– Consolidate and eliminate unsupported databases andversions
  41. 41. © 2012 IBM Corporation41 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementArchiveArchiveOptim™ value for Application RetirementOracle ERPDB2 Order EntryLegacyWMSOracle ERPOracle ERPDB2 Order EntryDB2 Order EntryLegacyWMSLegacyWMSComplete Business ObjectYear 2006Year 2005Year 2004Year ….….….<XML><Name>John</Name><Zip>08540</Zip></XML><XML><Name>John</Name><Zip>08540</Zip></XML>AccessAccessExpireRetainStoreDecommissionDecommissionOPTIM™ Solution
  42. 42. © 2012 IBM Corporation42 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementOPTIM™ Technical AdvantagesArchives are immutable–Data + Metadata +related documentsEnables compression–Smaller footprint–Simplifies operations,backup, restoreDevice independent–Direct access–SAN, NAS, CAS, WORM–Retrieve & restore to anyenterprise class databaseDatabase independent–“Future proofed”Supports HSM–As the data ages it can moveto less expensive storageand still be accessedDesigned with Data Agingpolicies in mind–Retention–ExpirationPart of consistent Optimstrategy for all databases andapps–Decommissioning–Archiving–TDMOPTIM™ Solution
  43. 43. © 2012 IBM Corporation43 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementAccess to Archived dataOpen Data Manager (ODM)Provides access to data in Archive Files / Archive Collections forapplications that use ODBC and JDBC APIsOptim ArchiveFilesODM Primary Server(Optim Directory & definedODM Data Sources)JDBC or ODBCaccessQuery /ReportingToolsMS-Excel, MS-Access,Crystal Reports, etc., thatUse ODBC/JDBC
  44. 44. © 2012 IBM Corporation44 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementArchive File CollectionsAn Archive Collection is a list ofArchive Files that can belogically unioned together as asingle data source for OpenData Manager (ODM) access.The Archive File CollectionEditor is used to create anArchive File CollectionTables with matching creatorIDs and names in separateArchive Files will be unionedUsing the Archive File Collection toaccess data in multiple ArchiveFiles:Even if all files do not include aspecific tableEven if all files do not contain aspecific columnEven if the attributes of data in acolumn vary from file to fileODM processes files in the orderthey are listed in the Archive FileCollection Editor
  45. 45. © 2012 IBM Corporation45 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementTechnical Features and Benefits of ODMODM features and benefits include:–Access archived data, with the ability to join tables, groupdata, perform unions, etc. under full SQL 92 viaODBC/JDBC–Output data in archive files as XML–Retain archive security for archive files and rows/columns.–Leverage ODM to access archived DB2 data—convertsOptim z/OS archive files to Optim (open system) archivefiles.–Perform queries and reports that confederate productionand archived data as though data was never relocated(supports connectors to a data source other than thearchive file).–Use standard reporting and query tools like Excel, Crystalreports and SQL utilities against archived data.
  46. 46. © 2012 IBM Corporation46 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation Management• Application access− User requests data with application function• Consolidated viewApplication-Based Access to ArchivesApplicationOLTPArchiveArchive
  47. 47. © 2012 IBM Corporation47 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementSelf-Help Access to ArchivesOpen & independentNo training of end users or audit staffLeverage existing tools and skillsOLTP not requiredExcelAccessReport Writers –• Cognos,• Business Objects,• DiscovererAny SQL-based tool setBrowserJava ApplicationArchiveArchive
  48. 48. © 2012 IBM Corporation48 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementApplication-Independent Access to ArchivesApplication not required– Enables decommissioning of obsolete or redundant applications andversions– No reliance on application vendorOLTP not required– Offline access available– Archive access does not impede production processingXML, SQL,ODBC / JDBCDelivered BrowserIndexed & SecuredCompanyContactActivityArchive
  49. 49. © 2012 IBM Corporation49 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementNon Production EnvironmentsSubset & MaskProduction EnvironmentsArchiveSingle, scalable, interoperable EDM solution provides a central point to deploy policies to extract, store, port, and protect applicationdata records from creation to deletionNAS SAN ATA CAS Optical TapeAIX Linux z/OS Windows XP/2000 Solaris HP/UXDB2 LUW DB2 z/OS DB2 iSeries Informix Sybase SQL Server IMS VSAM Adabas Oracle More ….Optim ™Data Growth, Data Privacy, Test Data Management, Application Upgrades, Application RetirementEnterprise Architecture
  50. 50. © 2012 IBM Corporation50 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation Management
  51. 51. © 2012 IBM Corporation51 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementUsing Discovery + Optim for Data Privacy
  52. 52. © 2012 IBM Corporation52 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementThe Problem: Securing InformationLocate: Confidential information is dispersed throughout the organization– Hard to locate and easily transmitted, copied and printedMonitor: Severe financial, business and regulatory penalties for loss ofsensitive and regulated data– Business challenges - Loss of reputation and customer confidence– Regulatory challenges - GLBA, SOX, HIPAAProtect: Difficult to set policies without the capability to monitor and enforce– Lack of automated enforcement mechanisms
  53. 53. © 2012 IBM Corporation53 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementSensitive Data Discovery: Where is all the Sensitive Data?How Long Will it Take to Find It?DiscoverSensitiveDataPoorly Documented Applications:… ? …
  54. 54. © 2012 IBM Corporation54 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementSensitive Data Discovery: Structured Data• Finding Sensitive Data Elements (SDE) in each system can take days• Whole and partial SDE’s can be found in hundreds of tables and fieldsR o w M e m b e r S S # A g e P h o n e S e x1 5 9 5 8 4 6 2 2 6 1 2 3 - 4 5 - 6 7 8 9 1 5 (1 2 3 ) 4 5 6 - 7 8 9 0 M2 5 6 7 4 7 2 5 9 6 1 3 8 - 2 7 - 1 6 0 4 8 (1 3 8 ) 2 7 1 - 6 0 3 7 F3 5 4 0 4 5 0 0 9 1 1 5 4 - 8 6 - 4 1 9 6 2 2 (1 5 4 ) 8 6 4 - 1 9 6 1 M4 5 1 4 7 1 4 3 7 2 1 7 3 - 4 4 - 7 9 0 0 5 5 (1 7 3 ) 4 4 7 - 8 9 9 6 F5 4 9 0 2 0 4 1 6 4 1 9 4 - 2 6 - 1 6 4 8 4 (1 9 4 ) 2 6 1 - 6 4 7 6 F6 4 6 6 8 6 1 1 0 9 2 1 7 - 5 7 - 3 0 4 6 6 6 (2 1 7 ) 5 7 3 - 0 4 5 3 M9 8 7 ,6 2 3 4 4 4 6 2 9 6 2 8 2 4 3 - 6 8 - 1 8 1 2 2 5 (2 4 3 ) 6 8 1 - 8 1 0 7 F9 8 7 ,6 2 4 4 2 3 4 5 6 7 8 9 2 7 2 - 9 2 - 3 6 2 9 8 7 (2 7 2 ) 9 2 3 - 6 2 8 0 MK n o w n S e n s itiv e D a taSensitive Data Repository
  55. 55. © 2012 IBM Corporation55 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementWhy is Sensitive Data Discovery in Structured Data Difficult?Patient Result Test3802468 N 534182715 N 534600986 N 325061085 N 535567193 N 726123913 Y 476736304 N 347409934 N 348150928 N 478966020 N 34System A Table 15Sensitive Relationship Discovery Sensitive data can’t be found just by a simple data scan.– Must connect tables and lookup tables– Hidden within larger fields (substring)– Hidden across fields (concatenations)– Represented differently (lookup tables and case statements)“Corporate memory” is poor– Documentation is incomplete– SME’s and Data Analysts are only knowledgeable of one ortwo systemsHundreds of tables and millions of rows:– Complex– Difficult to verifyData quality problems make discovery even more difficultTest Name53 Streptococcus pyogenes72 Pregnancy32 Alzheimer Disease47 Hemorrhoids34 DermatamycosesSystem Z Table 25Number Name4600986 AlexFulltheim8150928 BarneySolo6736304 BillAlexander3802468 BobSmith5567193 EileenKratchman7409934 FredSimpson6123913 GregLougainis5061085 JamieSlattery4182715 JimJohnson8966020 MartinAstonSystem A Table 1Number Name3544600986 AlexFulltheim5728150928 BarneySolo3786736304 BillAlexander6783802468 BobSmith4035567193 EileenKratchman8037409934 FredSimpson4306123913 George Brett9525061085 JamieSlattery4594182715 JimJohnson1288966020 MartinAstonSystem A Table 1
  56. 56. © 2012 IBM Corporation56 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementInfoSphere Basic Discovery For Sensitive DataAnalyze up to 20 data sourcessimultaneouslyDiscover sensitive data by comparingknown sensitive data with data in a widevariety of systems at the push of abuttonIdentified sensitive data elements(SDEs) are exported to Optim formasking
  57. 57. © 2012 IBM Corporation57 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementTransformation Analyzer Summary for Hidden Sensitive DataAutomates discovery of complex businessrules between data sources– Finds sensitive data hidden withinlonger fields (e.g. SSN hidden in a 46digit routing number)– Finds sensitive data that has beendivided up across multiple columns (e.g.SSN divided into three separatecolumns)– Finds sensitive data that has beentransformed (i.e. items converted intocodes)
  58. 58. © 2012 IBM Corporation58 Discovering the Value of Optim for Test Data Management, Data Privacy and Data ArchivingInformation ManagementAutomated Sensitive Data AdvantagesBetter ROI – faster time to deployment at lower cost– 5-10x time savings for business object discovery– 5-10x time savings for identifying sensitive informationAutomatedSensitive DataDiscoveryCustomerSuccessOptimDeployment

×