Successfully reported this slideshow.
Champion & FAS Deduplication Overview & Best Practices For More Info please contact Michael Hudak  NetApp Sales Specialist...
FAS Deduplication <ul><li>GD Release April 2007 </li></ul><ul><li>R200 </li></ul><ul><li>FAS2000 </li></ul><ul><li>FAS3000...
Space Reduction Technologies 2002 1992 2004 2006 2004 - RAID-DP 2002 – SnapVault/OSSV 1993 – Snapshot Technology 2005 - Th...
Deduplication Basics: Fingerprint Catalog <ul><li>A deduplication catalog consists of a series of “hash values” aka digita...
Deduplication Basics: Reference Pointers <ul><ul><li>Data objects are written to storage systems using “Reference Pointers...
NetApp Enabling Technology: WAFL Block Sharing <ul><li>FAS deduplication utilizes block sharing within the WAFL file syste...
FAS Deduplication: Commands <ul><li>License it </li></ul><ul><li>Turn it on </li></ul><ul><li>[Deduplicate existing data] ...
FAS Deduplication: “ sis status ” Progress Messages and Stages Path  State  Status  Progress /vol/vol5  Enabled  Active  4...
Deduplication Space Savings <ul><li>Space Savings Will Vary Based On Data Types </li></ul><ul><li>Use NetApp Space Savings...
<ul><li>Scans volumes and discovers duplicate data </li></ul><ul><ul><li>Simulates the effect of FAS deduplication </li></...
Using SSET 2.0 <ul><li>Using the tool—command example: </li></ul><ul><ul><li>Find_space –f <fingerprint file> -p <path> </...
SSET 2.0 Example
Deduplication Best Practices: Qtree SnapMirror® (QSM) Replication <ul><li>QSM replication </li></ul><ul><li>Improves stora...
Deduplication Best Practices: Volume SnapMirror® (VSM) Replication <ul><li>VSM replication </li></ul><ul><li>Deduplicated ...
Deduplication Best Practices: Copying to Tape Third-Party Backup Application Server DB Server Deduplication  SAN/NAS NDMP ...
Deduplication Best Practices: Scheduling with Backup Data Third-Party Backup Application Server DB Server Deduplication  S...
Deduplication Best Practices: Scheduling with Archival Data Deduplication  Volume SnapMirror (VSM)  Volume SnapMirror® (VS...
Deduplication Best Practices: Scheduling Light-Duty Primary Data Mission-Critical Primary Storage “ Lite Use” Primary Stor...
Deduplication in a VMware® Infrastructure <ul><li>A VMware infrastructure consists of virtual machine (VM) templates and c...
Cloning a VMware® Virtual Machine <ul><li>VM templates and clones can grow very large, for example, one NetApp user with 1...
An Opportunity for Deduplication <ul><li>The creation of VM clone images presents an opportunity for space reduction via d...
Deduplication with VMware® VMs <ul><li>Space savings:  </li></ul><ul><ul><li>Up to 90% </li></ul></ul><ul><li>Deduplicatio...
Deduplication Miscellaneous Best Practices <ul><li>SnapVault®/OSSV </li></ul><ul><ul><li>Deduplicate only the baseline ima...
Volume Limits <ul><li>FAS Deduplication Volume Limits </li></ul>
Resources <ul><li>Deduplication FAQs -> </li></ul><ul><li>TR-3505—  Deduplication Deployment and Implementation Guide </li...
Upcoming SlideShare
Loading in …5
×

Champion Fas Deduplication

1,508 views

Published on

FAS Deduplication Overview and Best Practices

Published in: Technology, Business
  • Be the first to comment

Champion Fas Deduplication

  1. 1. Champion & FAS Deduplication Overview & Best Practices For More Info please contact Michael Hudak NetApp Sales Specialist [email_address] 800-771-7000 x344
  2. 2. FAS Deduplication <ul><li>GD Release April 2007 </li></ul><ul><li>R200 </li></ul><ul><li>FAS2000 </li></ul><ul><li>FAS3000 </li></ul><ul><li>FAS6000 </li></ul><ul><li>V-Series (2008) </li></ul><ul><li>Multi-tier Deduplication </li></ul><ul><li>Primary data </li></ul><ul><li>Backup data </li></ul><ul><li>Archival data </li></ul>NetApp Deduplication System Adoption 2007 Projection = 1,700 Systems Deduplication-Enabled System Storage = 50PB
  3. 3. Space Reduction Technologies 2002 1992 2004 2006 2004 - RAID-DP 2002 – SnapVault/OSSV 1993 – Snapshot Technology 2005 - Thin Provisioning 2006 - VTL Compression 2007 - Deduplication 2008 2001 – SATA NearStore 2006 – SnapVault for NBU 2005 – Virtual Cloning Cost/GB Time “ Additive” Space Reduction Features
  4. 4. Deduplication Basics: Fingerprint Catalog <ul><li>A deduplication catalog consists of a series of “hash values” aka digital fingerprints </li></ul><ul><li>Once catalogued, hashes can be compared and deduplication candidates identified </li></ul>Hashing Algorithm Data Object Digital Fingerprint Fingerprint Catalog
  5. 5. Deduplication Basics: Reference Pointers <ul><ul><li>Data objects are written to storage systems using “Reference Pointers” </li></ul></ul><ul><li>Deduplication introduces two important concepts: </li></ul><ul><ul><li>Catalog of data objects </li></ul></ul><ul><ul><li>The ability to reference one object multiple times </li></ul></ul>Non-Deduplicated Reference Pointers Allocated Storage Allocated Storage Allocated Storage Allocated Storage Allocated Storage Deduplication Catalog Deduplicated Reference Pointers Allocated Storage Free Storage Free Storage Free Storage Free Storage
  6. 6. NetApp Enabling Technology: WAFL Block Sharing <ul><li>FAS deduplication utilizes block sharing within the WAFL file system </li></ul><ul><li>A single block can be referenced up to 255 times </li></ul><ul><li>This technology has been in place for 15 years (Snapshots) </li></ul>INODE 1 INODE 2 IND IND IND IND DATA DATA DATA DATA
  7. 7. FAS Deduplication: Commands <ul><li>License it </li></ul><ul><li>Turn it on </li></ul><ul><li>[Deduplicate existing data] </li></ul><ul><li>Schedule when to deduplicate or run manually </li></ul><ul><li>Check out what’s happening </li></ul><ul><li>See the savings! </li></ul><ul><ul><li>license add <a_sis> </li></ul></ul><ul><ul><li>sis on <vol> </li></ul></ul><ul><ul><li>sis start -s <vol> </li></ul></ul><ul><ul><li>sis config [-s schedule] <vol> </li></ul></ul><ul><ul><li>sis start <vol> </li></ul></ul><ul><ul><li>sis status [-l] <vol> </li></ul></ul><ul><ul><li>df – s <vol> </li></ul></ul>
  8. 8. FAS Deduplication: “ sis status ” Progress Messages and Stages Path State Status Progress /vol/vol5 Enabled Active 40MB (20%) done Path State Status Progress /vol/vol5 Enabled Active 30MB Verified OR /vol/vol5 Enabled Active 10% Merged Filer> sis status Path State Status Progress /vol/vol5 Enabled Active 25 MB Scanned Path State Status Progress /vol/vol5 Enabled Active 25 MB Searched Gathering Sorting Deduplicating Checking
  9. 9. Deduplication Space Savings <ul><li>Space Savings Will Vary Based On Data Types </li></ul><ul><li>Use NetApp Space Savings Estimation Tool (SSET) For Validation </li></ul>
  10. 10. <ul><li>Scans volumes and discovers duplicate data </li></ul><ul><ul><li>Simulates the effect of FAS deduplication </li></ul></ul><ul><li>Does not require Data ONTAP® or A-SIS license </li></ul><ul><li>Three standalone executables: </li></ul><ul><ul><li>Linux® </li></ul></ul><ul><ul><li>Solaris™ </li></ul></ul><ul><ul><li>Windows® </li></ul></ul><ul><li>Available from NetApp and Partner SE’s </li></ul>SSET 2.0 Overview
  11. 11. Using SSET 2.0 <ul><li>Using the tool—command example: </li></ul><ul><ul><li>Find_space –f <fingerprint file> -p <path> </li></ul></ul><ul><li>Tool will “crawl” through the path specified and create fingerprints for each block of data </li></ul><ul><li>Fingerprints are compared and matches are reported </li></ul><ul><li>2TB maximum; if tool determines that the path is >2TB, will exit with error message </li></ul><ul><li>Large volumes will take a long time to analyze </li></ul><ul><li>Tool should not be left installed at customer site once the evaluation is completed </li></ul>
  12. 12. SSET 2.0 Example
  13. 13. Deduplication Best Practices: Qtree SnapMirror® (QSM) Replication <ul><li>QSM replication </li></ul><ul><li>Improves storage efficiency at secondary location </li></ul><ul><li>No impact on primary storage workload </li></ul><ul><li>V-Series data can be mirrored to DR site and also deduplicated </li></ul>FAS at Site A, e.g., Data Center FAS at Site B, e.g., DR Site deduplication QSM V-Series
  14. 14. Deduplication Best Practices: Volume SnapMirror® (VSM) Replication <ul><li>VSM replication </li></ul><ul><li>Deduplicated data at primary and secondary locations </li></ul><ul><ul><li>Secondary site inherits deduplicated data </li></ul></ul>FAS at Site A, e.g., Data Center Network Efficiency Reduced amount of data traveling across the network FAS at Site B, e.g., DR Site VSM Deduplication V-Series (Q1 2008) Deduplication “ Inherited” deduplication
  15. 15. Deduplication Best Practices: Copying to Tape Third-Party Backup Application Server DB Server Deduplication SAN/NAS NDMP To Tape NDMP to tape can be accomplished at any time - No need to wait for deduplication to complete Primary Storage ERP/ECM Server E-mail Server
  16. 16. Deduplication Best Practices: Scheduling with Backup Data Third-Party Backup Application Server DB Server Deduplication SAN/NAS Volume SnapMirror (VSM) Volume SnapMirror® (VSM) Deduped image is mirrored Saves network bandwidth and storage space on both NearStore units DR Site Deduplication Scripted after Each Backup: sis start <vol> Primary Storage ERP/ECM Server E-mail Server
  17. 17. Deduplication Best Practices: Scheduling with Archival Data Deduplication Volume SnapMirror (VSM) Volume SnapMirror® (VSM) Deduped image is mirrored Saves network bandwidth and storage space on both NearStore units DR Site Deduplication Automated Schedule Based on 20% Change Rate: sis config –s auto <vol> Third-Party Archival Application Server SAN/NAS Primary Storage ERP/ECM Server E-mail Server
  18. 18. Deduplication Best Practices: Scheduling Light-Duty Primary Data Mission-Critical Primary Storage “ Lite Use” Primary Storage Servers Clients Deduplication VMware ®, CIFS shares, home dirs, etc. Volume SnapMirror (VSM) Volume SnapMirror® (VSM) Deduped image is mirrored Saves network bandwidth and storage space on both NearStore units DR Site Deduplication Scheduled during Off-Peak Time: sis config –s schedule <vol> SAN/NAS
  19. 19. Deduplication in a VMware® Infrastructure <ul><li>A VMware infrastructure consists of virtual machine (VM) templates and clone copies </li></ul><ul><li>Templates, or Golden Masters, are created for each application environment and consist of a VM configuration file (.vmx) and one or more virtual disk files (.vmdk) </li></ul>
  20. 20. Cloning a VMware® Virtual Machine <ul><li>VM templates and clones can grow very large, for example, one NetApp user with 1,800 VMs requires 64TB of disk capacity to manage these copies </li></ul>Virtual Machines ESX Server
  21. 21. An Opportunity for Deduplication <ul><li>The creation of VM clone images presents an opportunity for space reduction via deduplication </li></ul><ul><li>Deduplication removes redundant blocks within a NetApp system volume and does so in a transparent manner so that all clone copies appear intact to the ESX server </li></ul>Virtual Machines ESX Server Deduplication
  22. 22. Deduplication with VMware® VMs <ul><li>Space savings: </li></ul><ul><ul><li>Up to 90% </li></ul></ul><ul><li>Deduplication runs as background task, scheduled during off-peak times </li></ul><ul><li>Deduplication imposes only nominal impact on read/write performance </li></ul>Deduplication Up to 90% Space Savings Remote Data Center VMware ESX Servers SAN/ NAS Primary Data Center “ Golden” VMware Masters + Virtual Machine Clones NetApp FAS System SnapMirror® Replication SAN/ NAS Up to 90% Space Savings NetApp FAS System
  23. 23. Deduplication Miscellaneous Best Practices <ul><li>SnapVault®/OSSV </li></ul><ul><ul><li>Deduplicate only the baseline image today </li></ul></ul><ul><ul><li>Extended use will be supported in Data ONTAP 7.3 </li></ul></ul><ul><li>Snapshot™ Copies </li></ul><ul><ul><ul><li>Deduplicate before taking Snapshot copies </li></ul></ul></ul><ul><ul><ul><li>Delete stale Snapshot copies </li></ul></ul></ul><ul><ul><ul><li>Refer to Deployment Guide for detailed info </li></ul></ul></ul><ul><ul><ul><li>Efficiency will improve in Data ONTAP 7.3 </li></ul></ul></ul>
  24. 24. Volume Limits <ul><li>FAS Deduplication Volume Limits </li></ul>
  25. 25. Resources <ul><li>Deduplication FAQs -> </li></ul><ul><li>TR-3505— Deduplication Deployment and Implementation Guide </li></ul><ul><li>Online Backup and Recovery Guide </li></ul><ul><li>Space Savings Estimation Tool </li></ul><ul><li>All Resources: </li></ul><ul><ul><li>PartnerCenter>Products>NearStore on FAS </li></ul></ul>

×