Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Paul Angus – Backup & Recovery in CloudStack

685 views

Published on

CloudStack users have long had to resort to using snapshots as a form of VM backup with varying success. In this talk Paul will explain features of the forthcoming backup and recovery feature.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Paul Angus – Backup & Recovery in CloudStack

  1. 1. The Cloud Specialists CloudStack Backup and Recovery Framework ShapeBlue.com • @ShapeBlue Paul Angus, VP Technology • @CloudyAngus paul.angus@shapeblue.com A P r e vi e w o f
  2. 2. The Cloud Specialists ShapeBlue.com @ShapeBlue “ShapeBlue are expert builders of public & private clouds. They are the leading independent global CloudStack services company” Ab o u t S h a p e B l u e
  3. 3. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Paul Angus – VP Technology • Global authority on CloudStack & cloud infrastructure design. • 20+ years C-Level experience. • Apache CloudStack project committer & PMC member • Specialising in deployment of CloudStack and surrounding infrastructure especially the user story • USP, Georgian Ministry of Justice, Orange, TomTom, PaddyPower, Ascenty, BSkyB, SAP, British Telecom Ab ou t m e
  4. 4. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
  5. 5. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
  6. 6. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
  7. 7. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue • What is a snapshot? (and what’s snapshots got to do with this anyway?) • Why is this a ‘framework’? (and why is it in quotes?) • What does this ‘framework’ do then? (enough with the quotes) • What will the framework work with? • An Example would be nice .. Cl ou d S t ac k B ac k u p an d Rec over y F r am ewor k
  8. 8. The Cloud Specialists ShapeBlue.com @ShapeBlue What’s is a Snapshot?
  9. 9. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue VM Snapshot • A hypervisor-driven point-in-time image of a virtual machine’s disks. The exact mechanism of this is dependant on the hypervisor. Volume Snapshot • A point-in-time image of a specific volume – The process usually involves taking a VM snapshot and then copying the required volume to secondary storage and the deleting the VM snapshot W h at ’ s a S n ap s h ot ?
  10. 10. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Features of volume snapshots include: • Alternate storage location of image (from primary storage) • Ability to schedule the volume snapshots • Ability to set a retention policy of n copies. • Ability to create new VMs from the snapshot image (pseudo restoration) • Ability to export volume snapshots from the system O K , b u t wh at ’ s s n ap s h ot s g ot t o d o wi t h t h i s ?
  11. 11. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue So snapshots are used by many (everyone) as a form of backup but… O K , b u t wh at ’ s s n ap s h ot s g ot t o d o wi t h t h i s ?
  12. 12. The Cloud Specialists ShapeBlue.com @ShapeBlue What’s wrong with snapshots as backups?
  13. 13. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue W h a t ’ s w r o n g w i t h s n a p s h o t s ? • Not consistent across volumes • A VM with separate disks for say; OS, logs and data, will have each disk processed at different times • Only crash consistent within a volume and they aren’t application aware • A database will not be quiesced before snapshot is taken • Transaction logs won’t be processed • VMware tools or XenTools may help with some basics
  14. 14. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue W h a t ’ s w r o n g w i t h s n a p s h o t s ? • They can be very slow. • Data has to be transferred from primary storage to secondary storage • In the case of vSphere, this data must go via the SSVM to be compressed into a OVA while being moved to secondary storage. • CloudStack locks out other actions while a snapshot is taking place. Workarounds have their own issues.
  15. 15. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue W h a t ’ s w r o n g w i t h s n a p s h o t s ? • Restoration of a VM requires a number of steps (and did I mention it’s slow) • User must make a template from snapshot (VM image copied via SSVM from one part of secondary storage to another – 2x network transfers) • User then creates VM from template (VM image copied from secondary to primary to create template on primary storage, then primary storage based template copied to make actual VM. 2x network assuming network based primary storage and 1x on-disk copy) • To restore a VM ‘exactly as it was’ requires considerable operator intervention
  16. 16. The Cloud Specialists ShapeBlue.com @ShapeBlue Why is this a ‘framework’
  17. 17. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue The CloudStack Backup & Recovery Framework • This is a framework because, by itself it doesn’t ‘do’ anything. • Plugins are required to drive physical actions • A dummy plugin has been created for testing the B&R framework API Oh. This is going to be a short/rubbish presentation then. W h y i s t h i s a ‘ f r a m e w o r k ’ ?
  18. 18. The Cloud Specialists ShapeBlue.com @ShapeBlue What does this ‘framework’ do then?
  19. 19. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue The CloudStack Backup and Recovery Framework Creates a vendor agnostic API and UI in CloudStack for end users to leverage 3rd party backup and recovery solutions. The B&R Framework abstracts the specifics of B&R solutions, such that through the use of a plugin, a 3rd party B&R solution can deliver …. W h at d o e s t h i s ‘ f r am ewor k ’ d o t h en ?
  20. 20. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue For the Cloud Operator: The B&R framework enables cloud operators to leverage their choice of enterprise grade backup and recovery technology. Giving them: • Greater control over the backend backup and recovery processes • Hardware and software accelerated performance • Reducing/elimination of operator overhead • More features to offer users • Additional chargeable services W h at d o e s t h i s ‘ f r am ewor k ’ d o t h en ?
  21. 21. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue For the Cloud Consumer: • Improved performance over snapshots • A feature set that consumers expect from backup software • Proper restoration options • Consistent API and user experience independent of B&R vendor • Piece of mind of a recognised industry named B&R solution W h at d o e s t h i s ‘ f r am ewor k ’ d o t h en ?
  22. 22. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Specifically: • Support for: • Operator created ‘policy based’ backups (RPOs) • Scheduled backups (similar to snapshot scheduling) • Ad hoc backups • Backups as first class citizens • ‘In-place’ restoration of entire VM including data disks • Restoration of VM backup after original VM is destroyed • Restoration of individual VM volumes (can then be attached to another VM) • Full integration into the CloudStack usage service, recording both protected and on-disk data metrics W h at d o e s t h i s ‘ f r am ewor k ’ d o t h en ?
  23. 23. The Cloud Specialists ShapeBlue.com @ShapeBlue What will the framework work with?
  24. 24. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Dummy Plugin • Built for compile time testing of the APIs. Veeam Backup & Replication 9.5 (for VMware) • Plugin written against Veeam Backup & Replication 9.5 and Apache CloudStack 4.12 • Tested against VMware 6.5 • Requires Veeam Backup & Replication Console, Veeam Backup Enterprise Manager, OpenSSH and PowerShell on the Veeam Server W h at w i l l t h e f r am ew or k w or k w i t h ?
  25. 25. The Cloud Specialists ShapeBlue.com @ShapeBlue Example(s) ?
  26. 26. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Backup APIs so far* • createVMBackup(A) • deleteBackupPolicydeleteVM • BackupimportBackupPolicy (A) • listBackupPolicies • listBackupProviders • listVMBackupRestorePoints • listVMBackupsrestoreVMBackup (A) • restoreVolumeFromBackup (A) • startVMBackup (A) *Still subject to change during development Th e B a c k u p an d Rec over y AP I
  27. 27. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Veeam B & R P l u g i n ‘Policy Based’ Backups • listBackupPolices external=true Returns ‘template’ jobs. • importBackupPolicy Creates a policy (with a name and description) in CloudStack and maps it to a Veeam template job. ie. GOLD – 6hr RPO SILVER – 24hr RPO BRONZE – 7 day RPO
  28. 28. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Veeam B & R P l u g i n ‘Policy Based’ Backups • Add VM to the policy using createVMBackup* Creates a Veeam backup job for the VM with the properties of the template job linked.
  29. 29. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Veeam B & R P l u g i n ‘Ad-hoc’ & Scheduled Backups • CloudStack will send command to carry out an ad-hoc run of the policy based job or create one as required. • Ad-hoc and scheduled jobs will be driven by CloudStack • One VM to one Veeam job mapping allows for simpler accounting and unified incremental backups
  30. 30. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Veeam B & R P l u g i n Listing Backups listVMBackups • Lists backup jobs for a given VM. Under Veeam approach, this is a one-to-one relationship listVMBackupRestorePoints* • Veeam thinks in terms of restore points as being the image of a VM at any point in time. This API lists the restore points available for a given VM backup job (VM) *This is quite Veeam centric and potentially confusing, as the overall feature is rounded out, the abstraction naming will be reviewed.
  31. 31. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue VM and Volume Restoration • Veeam ‘in-place’ restoration is more in-place-of than in-the-original-place • The Veeam plugin takes care of the fact the replacement VM is in an alternate location to the ‘original’ • Current implementation automatically attaches a restored volume to a VM (option to restore to secondary storage will be added) Veeam B & R P l u g i n
  32. 32. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Veeam B & R P l u g i n ‘Ad-hoc’ & Scheduled Backups • CloudStack will send command to carry out an ad-hoc run of the policy based job or create one as required. • Ad-hoc and scheduled jobs will be driven by CloudStack • One VM to one Veeam job mapping allows for simpler accounting and unified incremental backups
  33. 33. The Cloud Specialists ShapeBlue.com @ShapeBlue That’s all cool. Where is it?
  34. 34. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Framework & Veeam Plugin • Expect to see a PR in Q4 • APIs largely done • Work left: • Scheduling of backups • Ad-hoc backups • Changing of underlying policy • Deletion of backups/restore points UI • Expect to see a PR late Q4 Ti m e l i n e
  35. 35. The Cloud Specialists ShapeBlue.com @ShapeBlue Questions?
  36. 36. The Cloud Specialists CloudStack Backup and Recovery Framework ShapeBlue.com • @ShapeBlue Paul Angus, VP Technology • @CloudyAngus paul.angus@shapeblue.com A P r e vi e w o f

×