Pete Stryjewski, Senior Managing Consultant Images Services 4.1.1 Integral SDS and Storage Overview

IS Release and Other Information IS 4.1.1 was released January 2008 IS 4.1.2 was release November 2008 Plasmon currently in receivership. Looking for a equity partner Support through Kodak continues Currently there is media supply MSAR “WORM” being fast tracked.

IS 4.1.1 was released January 2008

IS 4.1.2 was release November 2008

Plasmon currently in receivership.

Looking for a equity partner

Support through Kodak continues

Currently there is media supply

MSAR “WORM” being fast tracked.

New Storage Features for IS 4.1.1 Integral Single Document Storage (ISDS) Second generation connectivity to Protected Storage Devices Tightly integrated into “commit path” Read only MSAR support Full surfaces can be located on “read-only” file systems. File system based Protected Storage devices can enforce retention at the volume level. New enhancements in the pipeline – may be accelerated

Integral Single Document Storage (ISDS)

Second generation connectivity to Protected Storage Devices

Tightly integrated into “commit path”

Read only MSAR support

Full surfaces can be located on “read-only” file systems.

File system based Protected Storage devices can enforce retention at the volume level.

New enhancements in the pipeline – may be accelerated

What is Integral SDS? Integral SDS is… New IS server-based architecture Reuses robust OSAR/MSAR processes Plug-in device drivers model SDS references stored in MKF locator database In Image Services transactional path New, self-describing SDS object format

Integral SDS is…

New IS server-based architecture

Reuses robust OSAR/MSAR processes

Plug-in device drivers model

SDS references stored in MKF locator database

In Image Services transactional path

New, self-describing SDS object format

Integral SDS Benefits Robust Increased system stability and reliability SDS references are tamper proof from IS client application Extensible As new WORM compliant storage devices come on the market, “plug-in” model allows for development of new drivers in parallel Improved HA/DR No longer need to monitor NLS and script any failover. Can now leverage IS internal processes to start, stop and fail over

Robust

Increased system stability and reliability

SDS references are tamper proof from IS client application

Extensible

As new WORM compliant storage devices come on the market, “plug-in” model allows for development of new drivers in parallel

Improved HA/DR

No longer need to monitor NLS and script any failover. Can now leverage IS internal processes to start, stop and fail over

Integral SDS Benefits Improved performance MKF faster; frees up relational database resources Server-side efficient, integrated process Scalable: up to 254 devices and 99 “workers” per Fully integrates with IS commit/migration process Improved administration and ease of use Improved supportability Documentation, Education, Error handling part of IS

Improved performance

MKF faster; frees up relational database resources

Server-side efficient, integrated process

Scalable: up to 254 devices and 99 “workers” per

Fully integrates with IS commit/migration process

Improved administration and ease of use

Improved supportability

Documentation, Education, Error handling part of IS

Integral SDS Capabilities Native support of SDS devices More IS functionality supported with SDS Multiple storage configuration support All CFS-IS use cases when IS configured to use SDS Backwards compatibility with NLS: Immediate or day forward support from NLS to Integral SDS

Native support of SDS devices

More IS functionality supported with SDS

Multiple storage configuration support

All CFS-IS use cases when IS configured to use SDS

Backwards compatibility with NLS: Immediate or day forward support from NLS to Integral SDS

Integral SDS Capabilities Native support of SDS Devices Plug-in Model - new drivers for Network Appliance/IBM N-Series Snaplock Sun StorageTek Compliance(5x20 NAS with Compliance Archiving Software) EMC Centera IBM Total Storage Tivoli DR (DR550) Standard IS Tools Configure through fn_edit, Xapex Doc_tool to monitor performance and manage devices Error handling with RSVPs, Info Messages, SNMP

Native support of SDS Devices

Plug-in Model - new drivers for

Network Appliance/IBM N-Series Snaplock

Sun StorageTek Compliance(5x20 NAS with Compliance Archiving Software)

EMC Centera

IBM Total Storage Tivoli DR (DR550)

Standard IS Tools

Configure through fn_edit, Xapex

Doc_tool to monitor performance and manage devices

Error handling with RSVPs, Info Messages, SNMP

Backward Compatibility with NLS: Do nothing with NLS SDS documents – Still can be retrieved SDS_tool can still be used to manage these objects SDS information is not secure in the DOCTABA Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCTABA table Copy SDS info from DOCTABA to DOCS The SDS info (CLIP_ID) is copied from DOCTABA to the DOCS table Can still use SDS_tool to manage NLS SDS objects. It uses the DOCTABA sds info Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCS table

Do nothing with NLS SDS documents –

Still can be retrieved

SDS_tool can still be used to manage these objects

SDS information is not secure in the DOCTABA

Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCTABA table

Copy SDS info from DOCTABA to DOCS

The SDS info (CLIP_ID) is copied from DOCTABA to the DOCS table

Can still use SDS_tool to manage NLS SDS objects. It uses the DOCTABA sds info

Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCS table

Backward Compatibility with NLS (continued): Move SDS info from DOCTABA to DOCS Can not use SDS_tool to manage object New Integral SDS tools also may not be used Beware: Objects can not be managed and is read-only Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCS table Copy NLS object to new SDS unit The new Integral SDS sds_copy tool can be used to copy NLS SDS objects to a new Integral SDS unit New Integral SDS tools can be used to manage these new objects Note All the above modes and Integral SDS can co-exist on a single system

Move SDS info from DOCTABA to DOCS

Can not use SDS_tool to manage object

New Integral SDS tools also may not be used

Beware: Objects can not be managed and is read-only

Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCS table

Copy NLS object to new SDS unit

The new Integral SDS sds_copy tool can be used to copy NLS SDS objects to a new Integral SDS unit

New Integral SDS tools can be used to manage these new objects

Note

All the above modes and Integral SDS can co-exist on a single system

Device Specific Concerns Centera (all platforms) An “Advanced Retention Management” (ARM) License is required to be purchased from EMC. Provided Event Based Retention and Litigation Holds SnapLock/N-Series (Windows) Requires “domain” user to access mounted volume (like MSAR currently requires) IBM DR550 Retention cannot be increased

Centera (all platforms)

An “Advanced Retention Management” (ARM) License is required to be purchased from EMC.

Provided Event Based Retention and Litigation Holds

SnapLock/N-Series (Windows)

Requires “domain” user to access mounted volume (like MSAR currently requires)

IBM DR550

Retention cannot be increased

New SDS fields in the MKF Document Locator (DOCS) table: sds_id: signed long; -- SDS UNIT ID sds_obj_date: datetime; -- unix time at which SDS object was sds_obj_id_len: unsigned byte; sds_obj_id: hex byte [sds_obj_id_len:256]; sds_obj_blob_sz: unsigned long; sds_orig_doc_id: unsigned long -- This could be a foreign document. nonunique key; -- Nil value would indicate local document. sds_retent_typ: unsigned byte; -- Possible values -- None = 1 -- infinite = 2 -- chronological = 3 -- EBR + EBR supported device = 4 -- EBR + non-EBR supported device = 5 sds_retent_value: datetime; -- This retention expiration date. -- Not set for none, infinite, and EBR type -- Always set for chronological & virtual EBR sds_held: unsigned byte; -- If set to 1, it is held. sds_migrating: unsigned byte; -- If set to 1, it is migrating to SDS

sds_id: signed long; -- SDS UNIT ID

sds_obj_date: datetime; -- unix time at which SDS object was

sds_obj_id_len: unsigned byte;

sds_obj_id: hex byte [sds_obj_id_len:256];

sds_obj_blob_sz: unsigned long;

sds_orig_doc_id: unsigned long -- This could be a foreign document.

nonunique key; -- Nil value would indicate local document.

sds_retent_typ: unsigned byte; -- Possible values

-- None = 1

-- infinite = 2

-- chronological = 3

-- EBR + EBR supported device = 4

-- EBR + non-EBR supported device = 5

sds_retent_value: datetime; -- This retention expiration date.

-- Not set for none, infinite, and EBR type

-- Always set for chronological & virtual EBR

sds_held: unsigned byte; -- If set to 1, it is held.

sds_migrating: unsigned byte; -- If set to 1, it is migrating to SDS

Document Retention Really for the first time “retention matters” ‘ deldocs’ requires DOCTABA AND ISDS object retention to be met. This can be over-ridden. Individual tools (IDM desktop, WAL apps), can remove the “pointer” information and leave an orphaned object. ISDS objects (having met their retention) can be deleted via DOC_tool Backfile conversions should have proper retention set prior to migration.

Really for the first time “retention matters”

‘ deldocs’ requires DOCTABA AND ISDS object retention to be met. This can be over-ridden.

Individual tools (IDM desktop, WAL apps), can remove the “pointer” information and leave an orphaned object.

ISDS objects (having met their retention) can be deleted via DOC_tool

Backfile conversions should have proper retention set prior to migration.

SDS Retention Image Services retention has always been stored in the DOCTABA table With Integral SDS retention enabled, the retention stored in the DOCTABA is propagated to the SDS object and recorded in the DOCS table Retention of an the SDS object is inherited from the DCL when SDS objects are created as a result of committal Retention of an the SDS object is inherited from the DOCTABA when SDS objects are created as a result of migration (MSAR=>SDS migration) The sds_update_retention tool may be used to sync up SDS object with the DOCTABA (EBR object is triggered). Note it is not automatically done in 4.1.1.

Image Services retention has always been stored in the DOCTABA table

With Integral SDS retention enabled, the retention stored in the DOCTABA is propagated to the SDS object and recorded in the DOCS table

Retention of an the SDS object is inherited from the DCL when SDS objects are created as a result of committal

Retention of an the SDS object is inherited from the DOCTABA when SDS objects are created as a result of migration (MSAR=>SDS migration)

The sds_update_retention tool may be used to sync up SDS object with the DOCTABA (EBR object is triggered). Note it is not automatically done in 4.1.1.

Other Integral SDS features: Retrieval re-direction SDS to MSAR/optical (primary/tranlog) MSAR/optical (primary/tranlog) to SDS SDS is integrated into auto-restart write request infrastructure When Image Services is re-cycle, the write request(s) are restarted automatically Writes to SDS are guaranteed to happen SDS_worker reads FBC objects directly from cache SDS_worker knows how to process FBC objects efficiently. SDS_worker reads from MSAR surfaces directly when migrating from MSAR to SDS. When this is done, SDS_worker does not use page cache. SDS always create documents with checksums . If a checksum was not created during capture (no checksum in the long descriptor), Integral SDS will calculate checksum and write it at end of the SDS BLOB object

Retrieval re-direction

SDS to MSAR/optical (primary/tranlog)

MSAR/optical (primary/tranlog) to SDS

SDS is integrated into auto-restart write request infrastructure

When Image Services is re-cycle, the write request(s) are restarted automatically

Writes to SDS are guaranteed to happen

SDS_worker reads FBC objects directly from cache

SDS_worker knows how to process FBC objects efficiently.

SDS_worker reads from MSAR surfaces directly when migrating from MSAR to SDS.

When this is done, SDS_worker does not use page cache.

SDS always create documents with checksums .

If a checksum was not created during capture (no checksum in the long descriptor), Integral SDS will calculate checksum and write it at end of the SDS BLOB object

Lab Services Packages Integral SDS Installation Package - PS0064 “ Day forward” configuration Integral SDS Migration Mentoring Package – PS0065 “ Backfile” historical document migration (OSAR/MSAR) Conversion from NLS to ISDS

Integral SDS Installation Package - PS0064

“ Day forward” configuration

Integral SDS Migration Mentoring Package – PS0065

“ Backfile” historical document migration (OSAR/MSAR)

Conversion from NLS to ISDS

Questions? Pete Stryjewski - Lab Services Platform Services [email_address] Craig Werner – Lab Services Sales Manager [email_address] (Office) 916-641-445

Pete Stryjewski - Lab Services Platform Services

[email_address]

Craig Werner – Lab Services Sales Manager

[email_address]

(Office) 916-641-445