Pete Stryjewski, Senior Managing Consultant Images Services 4.1.1 Integral SDS and Storage Overview
IS Release and Other Information  <ul><li>IS 4.1.1 was released January 2008 </li></ul><ul><li>IS 4.1.2 was release Novemb...
New Storage Features for IS 4.1.1 <ul><li>Integral Single Document Storage (ISDS) </li></ul><ul><ul><li>Second generation ...
What is Integral SDS? <ul><li>Integral SDS is… </li></ul><ul><ul><li>New IS server-based architecture  </li></ul></ul><ul>...
Integral SDS Benefits <ul><li>Robust </li></ul><ul><ul><li>Increased system stability and reliability  </li></ul></ul><ul>...
Integral SDS Benefits <ul><li>Improved performance </li></ul><ul><ul><li>MKF faster; frees up relational database resource...
Integral SDS Capabilities <ul><li>Native support of SDS devices </li></ul><ul><li>More IS functionality supported with SDS...
Integral SDS Capabilities <ul><li>Native support of SDS Devices </li></ul><ul><ul><li>Plug-in Model - new drivers for </li...
Backward Compatibility with NLS: <ul><li>Do nothing with NLS SDS documents –  </li></ul><ul><ul><li>Still can be retrieved...
Backward Compatibility with NLS (continued): <ul><li>Move SDS info from DOCTABA to DOCS </li></ul><ul><ul><li>Can not use ...
Device Specific Concerns <ul><li>Centera (all platforms) </li></ul><ul><ul><li>An “Advanced Retention Management” (ARM) Li...
New SDS fields in the MKF Document Locator (DOCS) table: <ul><li>sds_id:  signed long;  -- SDS UNIT ID </li></ul><ul><li>s...
Document Retention <ul><li>Really for the first time “retention matters” </li></ul><ul><ul><li>‘ deldocs’ requires DOCTABA...
SDS Retention <ul><li>Image Services retention has always been stored in the DOCTABA table </li></ul><ul><li>With Integral...
Other Integral SDS features: <ul><li>Retrieval re-direction </li></ul><ul><ul><li>SDS to MSAR/optical (primary/tranlog) </...
Lab Services Packages <ul><li>Integral SDS Installation Package  - PS0064 </li></ul><ul><ul><li>“ Day forward” configurati...
Questions? <ul><li>Pete Stryjewski  - Lab Services Platform Services </li></ul><ul><ul><li>[email_address] </li></ul></ul>...
Upcoming SlideShare
Loading in …5
×

Images Services 4.1.1

2,881 views

Published on

Images Services 4.1.1
Integral SDS and Storage Overview

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Images Services 4.1.1

  1. 1. Pete Stryjewski, Senior Managing Consultant Images Services 4.1.1 Integral SDS and Storage Overview
  2. 2. IS Release and Other Information <ul><li>IS 4.1.1 was released January 2008 </li></ul><ul><li>IS 4.1.2 was release November 2008 </li></ul><ul><li>Plasmon currently in receivership. </li></ul><ul><ul><li>Looking for a equity partner </li></ul></ul><ul><ul><li>Support through Kodak continues </li></ul></ul><ul><ul><li>Currently there is media supply </li></ul></ul><ul><li>MSAR “WORM” being fast tracked. </li></ul>
  3. 3. New Storage Features for IS 4.1.1 <ul><li>Integral Single Document Storage (ISDS) </li></ul><ul><ul><li>Second generation connectivity to Protected Storage Devices </li></ul></ul><ul><ul><li>Tightly integrated into “commit path” </li></ul></ul><ul><li>Read only MSAR support </li></ul><ul><ul><li>Full surfaces can be located on “read-only” file systems. </li></ul></ul><ul><ul><li>File system based Protected Storage devices can enforce retention at the volume level. </li></ul></ul><ul><ul><li>New enhancements in the pipeline – may be accelerated </li></ul></ul>
  4. 4. What is Integral SDS? <ul><li>Integral SDS is… </li></ul><ul><ul><li>New IS server-based architecture </li></ul></ul><ul><ul><li>Reuses robust OSAR/MSAR processes </li></ul></ul><ul><ul><li>Plug-in device drivers model </li></ul></ul><ul><ul><li>SDS references stored in MKF locator database </li></ul></ul><ul><ul><li>In Image Services transactional path </li></ul></ul><ul><ul><li>New, self-describing SDS object format </li></ul></ul>
  5. 5. Integral SDS Benefits <ul><li>Robust </li></ul><ul><ul><li>Increased system stability and reliability </li></ul></ul><ul><ul><li>SDS references are tamper proof from IS client application </li></ul></ul><ul><li>Extensible </li></ul><ul><ul><li>As new WORM compliant storage devices come on the market, “plug-in” model allows for development of new drivers in parallel </li></ul></ul><ul><li>Improved HA/DR </li></ul><ul><ul><li>No longer need to monitor NLS and script any failover. Can now leverage IS internal processes to start, stop and fail over </li></ul></ul>
  6. 6. Integral SDS Benefits <ul><li>Improved performance </li></ul><ul><ul><li>MKF faster; frees up relational database resources </li></ul></ul><ul><ul><li>Server-side efficient, integrated process </li></ul></ul><ul><ul><li>Scalable: up to 254 devices and 99 “workers” per </li></ul></ul><ul><ul><li>Fully integrates with IS commit/migration process </li></ul></ul><ul><li>Improved administration and ease of use </li></ul><ul><li>Improved supportability </li></ul><ul><ul><li>Documentation, Education, Error handling part of IS </li></ul></ul>
  7. 7. Integral SDS Capabilities <ul><li>Native support of SDS devices </li></ul><ul><li>More IS functionality supported with SDS </li></ul><ul><ul><li>Multiple storage configuration support </li></ul></ul><ul><ul><li>All CFS-IS use cases when IS configured to use SDS </li></ul></ul><ul><li>Backwards compatibility with NLS: Immediate or day forward support from NLS to Integral SDS </li></ul>
  8. 8. Integral SDS Capabilities <ul><li>Native support of SDS Devices </li></ul><ul><ul><li>Plug-in Model - new drivers for </li></ul></ul><ul><ul><ul><li>Network Appliance/IBM N-Series Snaplock </li></ul></ul></ul><ul><ul><ul><li>Sun StorageTek Compliance(5x20 NAS with Compliance Archiving Software) </li></ul></ul></ul><ul><ul><ul><li>EMC Centera </li></ul></ul></ul><ul><ul><ul><li>IBM Total Storage Tivoli DR (DR550) </li></ul></ul></ul><ul><ul><li>Standard IS Tools </li></ul></ul><ul><ul><ul><li>Configure through fn_edit, Xapex </li></ul></ul></ul><ul><ul><ul><li>Doc_tool to monitor performance and manage devices </li></ul></ul></ul><ul><ul><ul><li>Error handling with RSVPs, Info Messages, SNMP </li></ul></ul></ul>
  9. 9. Backward Compatibility with NLS: <ul><li>Do nothing with NLS SDS documents – </li></ul><ul><ul><li>Still can be retrieved </li></ul></ul><ul><ul><li>SDS_tool can still be used to manage these objects </li></ul></ul><ul><ul><li>SDS information is not secure in the DOCTABA </li></ul></ul><ul><ul><li>Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCTABA table </li></ul></ul><ul><li>Copy SDS info from DOCTABA to DOCS </li></ul><ul><ul><li>The SDS info (CLIP_ID) is copied from DOCTABA to the DOCS table </li></ul></ul><ul><ul><li>Can still use SDS_tool to manage NLS SDS objects. It uses the DOCTABA sds info </li></ul></ul><ul><ul><li>Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCS table </li></ul></ul>
  10. 10. Backward Compatibility with NLS (continued): <ul><li>Move SDS info from DOCTABA to DOCS </li></ul><ul><ul><li>Can not use SDS_tool to manage object </li></ul></ul><ul><ul><li>New Integral SDS tools also may not be used </li></ul></ul><ul><ul><li>Beware: Objects can not be managed and is read-only </li></ul></ul><ul><ul><li>Retrievals will be done with the SDS_*_reader from SDS info stored in the DOCS table </li></ul></ul><ul><li>Copy NLS object to new SDS unit </li></ul><ul><ul><li>The new Integral SDS sds_copy tool can be used to copy NLS SDS objects to a new Integral SDS unit </li></ul></ul><ul><ul><li>New Integral SDS tools can be used to manage these new objects </li></ul></ul><ul><li>Note </li></ul><ul><ul><li>All the above modes and Integral SDS can co-exist on a single system </li></ul></ul>
  11. 11. Device Specific Concerns <ul><li>Centera (all platforms) </li></ul><ul><ul><li>An “Advanced Retention Management” (ARM) License is required to be purchased from EMC. </li></ul></ul><ul><ul><li>Provided Event Based Retention and Litigation Holds </li></ul></ul><ul><li>SnapLock/N-Series (Windows) </li></ul><ul><ul><li>Requires “domain” user to access mounted volume (like MSAR currently requires) </li></ul></ul><ul><li>IBM DR550 </li></ul><ul><ul><li>Retention cannot be increased </li></ul></ul>
  12. 12. New SDS fields in the MKF Document Locator (DOCS) table: <ul><li>sds_id: signed long; -- SDS UNIT ID </li></ul><ul><li>sds_obj_date: datetime; -- unix time at which SDS object was </li></ul><ul><li>sds_obj_id_len: unsigned byte; </li></ul><ul><li>sds_obj_id: hex byte [sds_obj_id_len:256]; </li></ul><ul><li>sds_obj_blob_sz: unsigned long; </li></ul><ul><li>sds_orig_doc_id: unsigned long -- This could be a foreign document. </li></ul><ul><li>nonunique key; -- Nil value would indicate local document. </li></ul><ul><li>sds_retent_typ: unsigned byte; -- Possible values </li></ul><ul><li>-- None = 1 </li></ul><ul><li>-- infinite = 2 </li></ul><ul><li>-- chronological = 3 </li></ul><ul><li>-- EBR + EBR supported device = 4 </li></ul><ul><li>-- EBR + non-EBR supported device = 5 </li></ul><ul><li>sds_retent_value: datetime; -- This retention expiration date. </li></ul><ul><li>-- Not set for none, infinite, and EBR type </li></ul><ul><li>-- Always set for chronological & virtual EBR </li></ul><ul><li>sds_held: unsigned byte; -- If set to 1, it is held. </li></ul><ul><li>sds_migrating: unsigned byte; -- If set to 1, it is migrating to SDS </li></ul>
  13. 13. Document Retention <ul><li>Really for the first time “retention matters” </li></ul><ul><ul><li>‘ deldocs’ requires DOCTABA AND ISDS object retention to be met. This can be over-ridden. </li></ul></ul><ul><ul><li>Individual tools (IDM desktop, WAL apps), can remove the “pointer” information and leave an orphaned object. </li></ul></ul><ul><ul><li>ISDS objects (having met their retention) can be deleted via DOC_tool </li></ul></ul><ul><li>Backfile conversions should have proper retention set prior to migration. </li></ul>
  14. 14. SDS Retention <ul><li>Image Services retention has always been stored in the DOCTABA table </li></ul><ul><li>With Integral SDS retention enabled, the retention stored in the DOCTABA is propagated to the SDS object and recorded in the DOCS table </li></ul><ul><li>Retention of an the SDS object is inherited from the DCL when SDS objects are created as a result of committal </li></ul><ul><li>Retention of an the SDS object is inherited from the DOCTABA when SDS objects are created as a result of migration (MSAR=>SDS migration) </li></ul><ul><li>The sds_update_retention tool may be used to sync up SDS object with the DOCTABA (EBR object is triggered). Note it is not automatically done in 4.1.1. </li></ul>
  15. 15. Other Integral SDS features: <ul><li>Retrieval re-direction </li></ul><ul><ul><li>SDS to MSAR/optical (primary/tranlog) </li></ul></ul><ul><ul><li>MSAR/optical (primary/tranlog) to SDS </li></ul></ul><ul><li>SDS is integrated into auto-restart write request infrastructure </li></ul><ul><ul><li>When Image Services is re-cycle, the write request(s) are restarted automatically </li></ul></ul><ul><ul><li>Writes to SDS are guaranteed to happen </li></ul></ul><ul><li>SDS_worker reads FBC objects directly from cache </li></ul><ul><ul><li>SDS_worker knows how to process FBC objects efficiently. </li></ul></ul><ul><li>SDS_worker reads from MSAR surfaces directly when migrating from MSAR to SDS. </li></ul><ul><ul><li>When this is done, SDS_worker does not use page cache. </li></ul></ul><ul><li>SDS always create documents with checksums . </li></ul><ul><ul><li>If a checksum was not created during capture (no checksum in the long descriptor), Integral SDS will calculate checksum and write it at end of the SDS BLOB object </li></ul></ul>
  16. 16. Lab Services Packages <ul><li>Integral SDS Installation Package - PS0064 </li></ul><ul><ul><li>“ Day forward” configuration </li></ul></ul><ul><li>Integral SDS Migration Mentoring Package – PS0065 </li></ul><ul><ul><li>“ Backfile” historical document migration (OSAR/MSAR) </li></ul></ul><ul><ul><li>Conversion from NLS to ISDS </li></ul></ul>
  17. 17. Questions? <ul><li>Pete Stryjewski - Lab Services Platform Services </li></ul><ul><ul><li>[email_address] </li></ul></ul><ul><li>Craig Werner – Lab Services Sales Manager </li></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>(Office) 916-641-445 </li></ul></ul>

×