Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automating Backup & Archiving with AWS and CommVault

3,437 views

Published on

Are you looking to automate backup and archiving of your business-critical data workloads? Attend this session to understand key use cases, best practices, and considerations for protecting your data with AWS and CommVault. This session will feature lessons learned from CommVault customers that have: migrated onsite backup data into Amazon S3 to reduce hardware footprint and improve recoverability; implemented data-tiering and archived data in Amazon Glacier for long term retention and compliance; performed snapshot-based protection and recovery for applications running in Amazon EC2; and, provisioned and managed VMs in Amazon EC2.

Speaker: Chris Gondek, Principal Architect, CommVault Australia and New Zealand

Published in: Technology

Automating Backup & Archiving with AWS and CommVault

  1. 1. Automating Backup & Archiving with AWS and CommVault Chris Gondek Principal Architect, CommVault Australia and New Zealand
  2. 2. Physical Virtual Public Cloud Applications Servers Network Storage Private Cloud CommVault Simpana Software
  3. 3. Recovery Tier Snapshot / Replica Protection / Archive Tier Disk / Appliance Based Long Term Retention Tier Object / Cloud / Tape Based Software/IP IntelliSnap Data Replicator Hardware/Array Datacore SANsymphony-V Dell Compellent & EqualLogic EMC Isilon, VMAX / Symmetrix, VPLEX, VNX / Celerra & VNX / CLARiiON Fujitsu ETERNUS DX HDS HP 3PAR & EVA Huawei IBM SVC & XIV LSI NetApp Nimble Storage Oracle ZFS SAN/DAS FC FoE iSCSI SCSI NAS CIFS NFS Appliance Data Domain Exagrid VTL Object / Cloud Amazon S3 (Native) Amazon S3 (Based) Amazon Glacier AT&T Synaptic Storage Caringo DDN WOS EMC Atmos (Native) EMC Atmos (Based) Hitachi Content Platform Microsoft Azure Storage OpenStack Object Storage (Native) OpenStack Object Storage (Based) Rackspace Cloud Files Tape For full support details refer to: http://documentation.commvault.com
  4. 4. Recovery Tier Snapshot / Replica Protection / Archive Tier Disk / Appliance Based Long Term Retention Tier Object / Cloud / Tape Based Real World Example: Global Investment Bank Data Age RestoreJobs
  5. 5. Recovery Tier Snapshot / Replica Protection / Archive Tier Disk / Appliance Based Long Term Retention Tier Object / Cloud / Tape Based 00 5,000 10,000 15,000 20,000 Less Than 7 Days 2 - 3 Weeks 1 - 2 Months 3 - 6 Months 9 - 12 Months Greater than 2 Years 2,396 1,384 4665629921,319 2,838 1,5881,716 3,547 19,447 Real World Example: Global Investment Bank Data Age RestoreJobs Number of Restore Jobs in a 12 Month Period Based on Data Age
  6. 6. Deployment Models Public Cloud Private Cloud Hybrid Cloud Deployment Models
  7. 7. Deployment Models TO AWS Cloud Data Management Use Cases DATA Public Cloud Private Cloud Hybrid Cloud Deployment Models
  8. 8. Deployment Models TO AWS Cloud Data Management Use Cases DATA IN AWS Cloud DATA Public Cloud Private Cloud Hybrid Cloud Deployment Models
  9. 9. Deployment Models TO AWS Cloud Data Management Use Cases DATA IN AWS Cloud DATA FROM AWS Cloud DATA Public Cloud Private Cloud Hybrid Cloud Deployment Models
  10. 10. AMI Amazon EBS Provisioni ng Lifecycle Management
  11. 11. AMI Amazon EBS Provisioni ng Snapshot Managem ent PIT Snapshot(s) Lifecycle Management
  12. 12. AMI Amazon EBS Amazon S3 Bucket Provisioni ng Snapshot Managem ent Backup & Archive PIT Snapshot(s) Lifecycle Management
  13. 13. AMI Amazon EBS Amazon S3 Bucket Bucket Amazon Glacier Provisioni ng Snapshot Managem ent Backup & Archive Vaulting PIT Snapshot(s) Lifecycle Management
  14. 14. Traditional Server Client Amazon EC2 Client / Disk Cache CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux / Unix) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket S3 Bucket Cloud Storage Target (REST / BLOB Storage)Client Amazon S3 Support
  15. 15. Traditional Server Client Amazon EC2 Client / Disk Cache CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux / Unix) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket S3 Bucket Cloud Storage Target (REST / BLOB Storage)Client Amazon S3 Support Natively Supported = No Dedicated Appliances
  16. 16. Traditional Server Client Amazon EC2 Client / Disk Cache CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux / Unix) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket S3 Bucket Cloud Storage Target (REST / BLOB Storage)Client Amazon S3 Support
  17. 17. Traditional Server Client Amazon EC2 Client / Disk Cache Chunk File (Backup to Disk) Sfile (Deduplication) CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux / Unix) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket S3 Bucket Cloud Storage Target (REST / BLOB Storage) Data Pipe Stream / Job) Stream #1/2 / Job) Stream #2/2 / Job) Client Amazon S3 Support
  18. 18. Traditional Server Client Amazon EC2 Client / Disk Cache Chunk File (Backup to Disk) Sfile (Deduplication) CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux / Unix) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket S3 Bucket Cloud Storage Target (REST / BLOB Storage) Data Pipe Stream / Job) Stream #1/2 / Job) Stream #2/2 / Job) Client Amazon S3 Support
  19. 19. Traditional Server Client Amazon EC2 Client / Disk Cache Chunk File (Backup to Disk) Sfile (Deduplication) CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux / Unix) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket S3 Bucket Cloud Storage Target (REST / BLOB Storage) Data Pipe Stream / Job) Stream #1/2 / Job) Stream #2/2 / Job) Client Amazon S3 Support More Streams = Greater Performance!
  20. 20. Traditional Server Client Amazon EC2 Client / Disk Cache Chunk File (Backup to Disk) Sfile (Deduplication) CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux / Unix) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket S3 Bucket Cloud Storage Target (REST / BLOB Storage)Client Amazon S3 Support Data Pipe Stream / Job) Stream #1/2 / Job) Stream #2/2 / Job)
  21. 21. Traditional Server Client Amazon EC2 Client / Disk Cache Cloud Connector Cloud Connector Cloud Connector Chunk File (Backup to Disk) Sfile (Deduplication) CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux / Unix) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket S3 Bucket Cloud Storage Target (REST / BLOB Storage) Sub-File / BLOB (16MB Default) Upload Threads (2 Default) Client Amazon S3 Support Data Pipe Stream / Job) Stream #1/2 / Job) Stream #2/2 / Job)
  22. 22. Traditional Server Client Amazon EC2 Client / Disk Cache Cloud Connector Cloud Connector Cloud Connector Chunk File (Backup to Disk) Sfile (Deduplication) CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux / Unix) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket S3 Bucket Cloud Storage Target (REST / BLOB Storage) Sub-File / BLOB (16MB Default) Upload Threads (2 Default) Client Amazon S3 Support Data Pipe Stream / Job) Stream #1/2 / Job) Stream #2/2 / Job)
  23. 23. Traditional Server Client Amazon EC2 Client / Disk Cache Cloud Connector Cloud Connector Cloud Connector Chunk File (Backup to Disk) Sfile (Deduplication) CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux / Unix) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket S3 Bucket Cloud Storage Target (REST / BLOB Storage) Sub-File / BLOB (16MB Default) Upload Threads (2 Default) Client Amazon S3 Support Data Pipe Stream / Job) Stream #1/2 / Job) Stream #2/2 / Job) Tunable Parameters for Optimised Read / Write
  24. 24. Data Mover Disk Cache CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 Cloud Storage Target (REST / BLOB Storage) Amazon Glacier Glacier Vault Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Amazon Glacier Support
  25. 25. Data Mover Disk Cache CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 S3 Bucket #2 Cloud Storage Target (REST / BLOB Storage) Amazon Glacier Glacier Vault Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Amazon Glacier Support
  26. 26. Data Mover Disk Cache CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 S3 Bucket #2 Cloud Storage Target (REST / BLOB Storage) Amazon Glacier Glacier Vault Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Amazon Glacier Support
  27. 27. Data Mover Disk Cache CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 S3 Bucket #2 Cloud Storage Target (REST / BLOB Storage) Amazon Glacier Glacier Vault Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Amazon Glacier Support Create a new S3 Bucket with Glacier Lifecycle Policies Enabled
  28. 28. Data Mover Disk Cache CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 Cloud Storage Target (REST / BLOB Storage) Amazon Glacier Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Amazon Glacier Support Glacier Vault S3 Bucket #2
  29. 29. Data Mover Disk Cache Cloud Connector Cloud Connector Chunk File (Backup to Disk) Re-Hydration CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 Cloud Storage Target (REST / BLOB Storage) Sub-File / BLOB (16MB Default) Amazon Glacier Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Auxiliary Copy Amazon Glacier Support Glacier Vault S3 Bucket #2
  30. 30. Data Mover Disk Cache Cloud Connector Cloud Connector Chunk File (Backup to Disk) Re-Hydration CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 Cloud Storage Target (REST / BLOB Storage) Sub-File / BLOB (16MB Default) Amazon Glacier Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Auxiliary Copy Amazon Glacier Support Glacier Vault S3 Bucket #2
  31. 31. Data Mover Disk Cache Cloud Connector Cloud Connector Chunk File (Backup to Disk) Re-Hydration CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 Cloud Storage Target (REST / BLOB Storage) Sub-File / BLOB (16MB Default) Upload Threads (2 Default) Amazon Glacier Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Auxiliary Copy Amazon Glacier Support Glacier Vault S3 Bucket #2
  32. 32. Data Mover Disk Cache Cloud Connector Cloud Connector Chunk File (Backup to Disk) Re-Hydration CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 Cloud Storage Target (REST / BLOB Storage) Sub-File / BLOB (16MB Default) Upload Threads (2 Default) Amazon Glacier Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Auxiliary Copy Amazon Glacier Support Glacier Vault S3 Bucket #2
  33. 33. Data Mover Disk Cache Cloud Connector Cloud Connector Chunk File (Backup to Disk) Re-Hydration CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 Cloud Storage Target (REST / BLOB Storage) Sub-File / BLOB (16MB Default) Upload Threads (2 Default) Amazon Glacier Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Auxiliary Copy Amazon Glacier Support Glacier Vault S3 Bucket #2 Policy Based Copy Management
  34. 34. Data Mover Disk Cache Cloud Connector Cloud Connector Chunk File (Backup to Disk) Re-Hydration CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 Cloud Storage Target (REST / BLOB Storage) Sub-File / BLOB (16MB Default) Upload Threads (2 Default) Amazon Glacier Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Auxiliary Copy Amazon Glacier Support Glacier Vault S3 Bucket #2
  35. 35. Data Mover Disk Cache Cloud Connector Cloud Connector Chunk File (Backup to Disk) Re-Hydration CommVault Archive File Container Data Offset / Index CommVault Media Agent (Window / Linux ) - Deduplication Node - Media / Library Manager Memory Cache Amazon S3 S3 Bucket #1 Cloud Storage Target (REST / BLOB Storage) Sub-File / BLOB (16MB Default) Upload Threads (2 Default) Amazon Glacier Amazon Lifecycle Policies move objects from S3 bucket into Glacier archives Auxiliary Copy Amazon Glacier Support Glacier Vault S3 Bucket #2 Archive
  36. 36. Use Cases
  37. 37. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Internet CommVault CommServe (Master Server) Mobile Client Endpoint Protection Amazon S3 S3 Bucket user
  38. 38. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Internet CommVault CommServe (Master Server) Mobile Client Endpoint Protection Amazon S3 S3 Bucket user On-Demand Self Service CommVault infrastructure running in the AWS Cloud
  39. 39. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Internet CommVault CommServe (Master Server) Mobile Client Endpoint Protection Amazon S3 S3 Bucket user
  40. 40. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Internet CommVault CommServe (Master Server) Mobile Client Endpoint Protection Amazon S3 S3 Bucket user
  41. 41. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Internet CommVault CommServe (Master Server) Mobile Client Recovery to Amazon WorkSpaces Amazon S3 S3 Bucket user Amazon WorkSpaces
  42. 42. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Internet CommVault CommServe (Master Server) Mobile Client Recovery to Amazon WorkSpaces Amazon S3 S3 Bucket user Amazon WorkSpaces Remote Lock & Remote Wipe
  43. 43. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Internet CommVault CommServe (Master Server) Mobile Client Recovery to Amazon WorkSpaces Amazon S3 S3 Bucket user Amazon WorkSpaces
  44. 44. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Internet CommVault CommServe (Master Server) Mobile Client Recovery to Amazon WorkSpaces Amazon S3 S3 Bucket user Amazon WorkSpaces Maintain Productivity
  45. 45. Internet CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager CommVault CommServe (Master Server) Mobile Client File Sync Amazon S3 S3 Bucket user
  46. 46. Internet CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager CommVault CommServe (Master Server) Mobile Client File Sync Amazon S3 S3 Bucket user Mobile Client
  47. 47. Internet CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager CommVault CommServe (Master Server) Mobile Client File Sync Amazon S3 S3 Bucket user Mobile Client Synchronise Files / Folders
  48. 48. Internet CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager CommVault CommServe (Master Server) Mobile Client File Share Amazon S3 S3 Bucket user
  49. 49. Internet CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager CommVault CommServe (Master Server) Mobile Client File Share Amazon S3 S3 Bucket user users
  50. 50. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Amazon S3 Bucket Backup in the AWS Cloud CommVault CommServe (Master Server)
  51. 51. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Amazon S3 Bucket Backup in the AWS Cloud CommVault CommServe (Master Server) CommVault infrastructure running in the AWS Cloud
  52. 52. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Corporate / Provider Data Center Physical Server Amazon S3 Bucket Backup to the AWS Cloud CommVault CommServe (Master Server) Direct Connect
  53. 53. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Corporate / Provider Data Center Physical Server Amazon S3 Bucket Backup to the AWS Cloud CommVault CommServe (Master Server) Onsite / Local Copy (Cache) Direct Connect
  54. 54. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Corporate / Provider Data Center Physical Server Backup to the AWS Cloud CommVault CommServe (Master Server) Amazon S3 Bucket Direct Connect
  55. 55. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Corporate / Provider Data Center Physical Server Backup to the AWS Cloud CommVault CommServe (Master Server) Offsite / DR Copy Amazon S3 Bucket Direct Connect
  56. 56. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Corporate / Provider Data Center Physical Server Vaulting to Glacier CommVault CommServe (Master Server) Amazon S3 Bucket Bucket Amazon Glacier Direct Connect
  57. 57. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Corporate / Provider Data Center Physical Server Vaulting to Glacier CommVault CommServe (Master Server) Amazon S3 Bucket Bucket Amazon Glacier Policy Based Copy Management Direct Connect
  58. 58. From retaining this… Reduce LTR Costs with Reference Copy
  59. 59. From retaining this… Relevant Data to this… Reduce LTR Costs with Reference Copy Reduce up to 70% of retention costs
  60. 60. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Corporate / Provider Data Center Physical Server CommVault CommServe (Master Server) Amazon S3 Bucket Automated Provisioning and Recovery
  61. 61. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Corporate / Provider Data Center Physical Server CommVault CommServe (Master Server) Amazon S3 Bucket Automated Provisioning and Recovery
  62. 62. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Corporate / Provider Data Center Physical Server CommVault CommServe (Master Server) Amazon S3 Bucket Policy Based Provisioning Automated Provisioning and Recovery AMI
  63. 63. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Corporate / Provider Data Center Physical Server CommVault CommServe (Master Server) Amazon S3 Bucket Automated Provisioning and Recovery
  64. 64. CommVault Media Agent(s) (Window / Linux) - Deduplication Node - Media / Library Manager Corporate / Provider Data Center Physical Server CommVault CommServe (Master Server) Amazon S3 Bucket Increase Productivity, Reduce Costs & Downtime Automated Provisioning and Recovery
  65. 65. 
 Demo
 
 “2 Clicks to the Cloud with Commvault and AWS”
  66. 66. Data Protection Summary
  67. 67. End Point Protection Data Protection Summary
  68. 68. Disaster Recovery End Point Protection Data Protection Summary
  69. 69. Archiving Disaster Recovery End Point Protection Data Protection Summary
  70. 70. Archiving eDiscovery Disaster Recovery End Point Protection Data Protection Summary
  71. 71. Physical Virtual Public Cloud Applications Servers Network Storage Private Cloud Summary

×