Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OpenStack Tokyo Talk Application Data Protection Service

1,275 views

Published on

OpenStack Tokyo Talk about Hyper-visor based replication in OpenStack and Application Data Protection Service in Smaug ADPaaS OpenStack DPaaS.

Published in: Engineering
  • Be the first to comment

OpenStack Tokyo Talk Application Data Protection Service

  1. 1. OpenStack Summit Tokyo 2015 Wang Hao, Software Engineer, Huawei IT Product Line Eran Gampel, Cloud Chief Architect , Huawei European Research Center Oshrit Feder, IBM Research - Haifa Cloud DR Orchestration: Beyond volume replication
  2. 2. Agenda Why we need disaster recovery? Replication in Cinder Hypervisor-based DR ADPaaS: Project Smaug Demo
  3. 3. Why do we need disaster recovery? Customers want 24x7 service availability Hardware Failures Human Error Accidents and Natural Disasters
  4. 4. Cinder Volume Replication Backup Snapshot Existing Data Protection Mechanisms In OpenStack
  5. 5. Got version 2 of replication in Liberty release Improve and make it more widely usable by other backend devices. None driver supported yet Implemented for Juno release Upstream OS code merged Support to IBM Storwize/SVC driver Begin from Icehouse summit Design summit on volume replication Status of Replication in Cinder
  6. 6. The main use of volume replication is resiliency in presence of failures. OpenStack Storage Backend Storage Backend Cinder DC#1 DC#2 Data Replication Use Case of Replication OpenStack
  7. 7. 4 1 2 3 5 6 7 8 9 Create Volume Type Create Volume Schedule Backend Replication Setup Replication Pair Replication StatusPromote Replica Recover from Replica Fail-Back Test Replication v1.0: Workflow
  8. 8. 4 1 2 3 5 6 7 8 9 Create Volume Type Create Volume Schedule Backend Replication Driver Selects Target & Setup Replication Pair Replication Status via Driver Report Failover to Secondary via API Recover from Replica Enable/Disable Replication Query Volume Replication Targets Replication v2.0: Workflow
  9. 9. Hypervisor-Level Replication Software-based Alternative for replication
  10. 10. Hypervisor LevelHardware Level Replication Solution Types Case in point: Hardware vs. Hypervisor Volume Storage HW Hypervisor VM IO Mirroring Replication Agent Volume Storage HW Volume Storage HW Hypervisor VM Volume Storage HW Source Target Source Target
  11. 11. Production Site DR Site DR Manager DR Manager Host IO Mirror VM VM VM Storage hypervisor VRGOpenStack Host Write Agent Storage hypervisor VRG OpenStack WAN OpenStack® Component New Component Vendor Component Protected VM Control Path Data Path Another choice: Hypervisor DR
  12. 12. IO Commands IO Completion IO Capture Write as normal Write ACK IO replication Queue IO Forwarding ,Compression and Encryption IO cache, Decompression and Decryption Write ACK IO Completion Write Write ACK IO Parsing Production Site DR Site Guest OS IO Mirror VRG VRG Write Agent Hypervisor DR: IO Mirroring
  13. 13. Setup Connection with vRG Start CBT Data Replication Consistency Check Queue Data Replication Queue overflow CBT done Finished1.Host abnormal restart 2. Swap(re-protect) Stop Hypervisor DR: IO Mirroring State Machine
  14. 14. 4 1 2 3 5 6 7 8 Configure Hypervisor Create VMs Protected Group Protection Policy Replication Start Create Recovery PlanFail-Over Re-Protect Fail-Back Hypervisor DR: Simplified Workflow
  15. 15. Replication Type HW Array Replication Hypervisor Replication Multi-Vendor Hardware Agnostic   No Impact on Compute Performance   No Special Network/Storage Privileges   No Special Admin Skillset Required   Transparent Deduplication   Virtualization-Ready   Cross VM Consistency Grouping Support   Cross Array Consistency Group Support   Hypervisor DR: HW(Array) vs. Hypervisor
  16. 16. Multiple Use Cases, Multiple Protection Plans Users need to be able to Choose the right protection plan Vendors need a way to plug different implementations
  17. 17. One API To Rule Them All
  18. 18. Is Data == Storage? Data Protection Service
  19. 19. DPaaS Architecture DPaaS Service APIs (REST) Service APIs (REST) File-Level Restore Policy Verification Replication Cinder Controller iSCSI FusionStorage eBackup Swift Cinder-Volume Cinder-Backup Message Queue Message Queue Cinder-API Swift Nova Horizon eBackup Metadata Backend Metadata OpenStack® Component New Component Huawei/Commercial Product Tiering Future release
  20. 20. But… We want to protect Applications, Services, Resources…
  21. 21. Case in point: Typical 3-tier Cloud App
  22. 22. Case in point: Typical 3-tier Cloud App Volume Web Net Router SG Web Srv 1 Project Web Srv 2 Image SG App Net App Server DB Net DB Server Image Image Volume
  23. 23. Data>> We need to protect all resources Storage
  24. 24. Introducing Smaug Application Data Protection as a Service
  25. 25. Smaug: Mission Statement Formalize Application Data Protection in OpenStack APIs, Services, Plugins, … Be able to protect Any Resource in OpenStack (as well as their dependencies) Allow Diversity of vendor solutions, capabilities and implementations without compromising usability
  26. 26. Smaug: Highlights Open Architecture Vendors create plugins that implement Protection mechanisms for different OpenStack resources User perspective: Protect App Deployment Configure and manage custom protection plans on the deployed resources (topology, VMs, volumes, images, …) Admin perspective: Define Protectable Resources Decide what plugins protect which resources, what is available for the user Decide where users can protect their resources
  27. 27. How to protect? (Protection Plans) Smaug: Application Data Protection as a Service What is protected? (Protected Resources) Where to protect? (Protection Banks) What was protected? (Protection Transactions) Who protects? (Protection Providers) Plan API Protection Resource API Protection Transaction API Bank API Pluggable Plan Enforcer Service Resource Protection Service Bank Vault Resource Protection Plugin Orchestrate
  28. 28. Overview Swift S3 … What is protected? (Protected Resources) VM Image Topology Volume How to protect? (Protection Plans) Protection Plan Name ID Protected Resource Trigger Retries Bank Options Volume Protection Plugin Backup Replication SnapshotWho protects? (Protection Providers) VM Protection Plugin Image Protection Plugin Topology Protection Plugin Protect Restore Verify OptionSchema ResultsSchema Protection API Read Write Bank API Where to protect? (Protection Banks) Bank Vault Cinder Nova … What was protected? (Protection Transactions) Ledger ProtectionTransaction implements Manual Time Event
  29. 29. Help us Build Smaug – Join the project https://launchpad.net/smaug IRC (gampel) eran.gampel@huawei.com oshritf@il.ibm.com Download Link
  30. 30. Demo Time Video -- Application DR With IBM Cloud Manger References Paris summit talk & demo European FP7 ORBIT Research project IBM Cloud Manager with Openstack
  31. 31. Thanks

×