VCLOUDSYNC – HYBRID DISASTER
RECOVERY FOR OPENSTACK

5 Mar 2014
Vinothini Raju,
CloudCamp, Bangalore
SCOPE
Backups & Restores across public clouds (AWS & GCE) and private clouds (Openstack
+ VMWare)
VMWARE & OPENSTACK CO-OPETITION
Co-operation
1. Openstack using VMWare
2. Complement features - VMWare (live migration, DRS etc) & OpenStack (Object Storage for
backups)

Competition
1. Customers see Openstack as an alternative to VMWare
2. Migrate VMWare environment to a neutral environment
CURRENT VMWARE + OPENSTACK INTEGRATION
Grizzly
1. ESX Driver (no vMotion, DRS, HA) , Driver runs in 1 nova instance per ESXi host
2. VC Driver (vMotion, DRS, HA), Driver runs in 1 nova instance per ESXi cluster, abstracts cluster

Havana
1. “Multi-cluster” management, 1 nova instance for multiple ESXi clusters
2. Block storage volumes (Cinder) can be stored as VMDKs as VMFS datastores (Snapshots go to vCenter
datastore).
3. VOVA (VMWare Openstack Virtual Appliance) – packaged OpenStack vApp

Icehouse roadmap
1. Cinder replication across different OpenStack installations
2. VMware Datastore Storage Backend (use image location reference instead of migrating the images)
3. VM diagnostics
HAVANA VMWARE VCDRIVER
Glance Store
Glance Images
Glance Service

Nova VMs
Nova Service

Cinder Service
VCDriver

upload_image/
fetch_image

Swift Service

Cinder Volumes

Attach/Detach Vols

Swift Store

VCenter

ESXi Cluster

ESXi Cluster
Shared VMFS Datastore
VMWARE BACKUP & DISASTER RECOVERY - TODAY
1. Consolidate backup
VMDK disks are snapshotted and mounted to backup proxy server and copies made to
backup storage.

2. VMs as physical systems
Backup agent runs inside the VM and backs up data to backup servers or attach iSCSI tape
as media & copy

3. VMs as a file on ESXi
Backup agent runs on the ESXi console & backups the VMDKs as a file to backup server.
RELIABILITY & HA ?
Twitter blames two-hour failure on dual data-center crashes.
Rawashdeh, VP, Infrastructure Operations Engineering at Twitter.
called an "infrastructural double whammy.“
Earlier:

• managed hosting services from NTT America
• Amazon’s cloud computing services to store and deploy
images, including profile pictures
The flooded lobby at the Verizon site at 140
West Street during the height of the storm
surge Monday night. (Photo:Verizon)

Now:
Twitter has setup datacenters to cater to its IT needs
WHAT DOES THIS IMPLY ?
1. Distributed environment (not all eggs in one basket)
2. Reliable Backup technology
3. Diverse clouds & infrastructure for backups
NEED FOR OBJECT STORE BACKUP
Object Store [1] - Distributed data (3 sites) compared to RAID backups (3 copies +
Tape backup)
1. Capacity: 60% more efficient
2. Cost : 80 – 90 % cost saved
3. Reliability : 100 Million times more reliable
Swift:
 Scalable
 Durable
 In house deployments [no-vendor lockin, cost effective, less latency]
[1] Ref: http://www.youtube.com/watch?v=koJwVEDTeg0
PROPOSED SOLUTION
VMWare VCDriver + Object Store for backup
1. Backup vmdks (root, data & snapshots) on to Object Store (Swift/Amazon S3)
2. Restore vmdks to VMWare shared datastore
VMWARE VCDRIVER + OBJECT STORE FOR BACKUP
Swift Store

Glance Store
Glance Images

Swift Service

Glance Service

Cinder Volumes

Nova VMs
Nova Service

Cinder Service

Attach/Detach Vols

Google
storage

upload_image/
fetch_image

S3

Backup/restore
images +
volumes

VCDriver

VCenter

ESXi Cluster

ESXi Cluster
Shared VMFS Datastore
VMWARE VCDRIVER + OBJECT STORE FOR BACKUP
(CONTD.,)
(root disk, data disk, snapshot disk)
Restore

Backup
VCDriver

Swift Service

S3
Google
Storage
Distributed (multi-site),
reliable & HA storage

vCloudSync
(Backup As A
Service)

Glance Store

Shared VMFS
Datastore
VCLOUDSYNC
vCloudSync
 Backup VMDK to AWS S3 & Google storage (Offline backups)
 Restore VMDK back to VMWare
 Google agent to migrate VMs between VMWare & GCE (Centos & Ubuntu)

Roadmap
 Snapshot backup
 Integration with VCDriver/OpenStack
 AWS agent to migrate VMs between VMWare & AWS (Ubuntu)
WHY VCLOUDSYNC ?
+ Agentless backups
+ Optmized for cloud backups and no local backups
+ Optmized Chunk size
+ Multi-part backups
+ Simultaneous backups
+ Deduplication
+ Compression (Ratio - 10:8.9)
+ ~ 20% better upload performance compared to manual uploads
+ Agnostic to virtualization or cloud technologies - Backup As A Service Engine
+ Reliability – meta data based backups
THANKS !
@vinothiniraju

TROV IT Consulting & Services Pvt Ltd,
#532, 2nd Floor, AECS Layout, B Block,
Bangalore - 560037, India

http://trov.co.in

Telephone:+91-80-40990162, +91-80-28540594
E-mail: info@trov.co.in

vCloudSync – Hybrid disaster recovery for Openstack

  • 1.
    VCLOUDSYNC – HYBRIDDISASTER RECOVERY FOR OPENSTACK 5 Mar 2014 Vinothini Raju, CloudCamp, Bangalore
  • 2.
    SCOPE Backups & Restoresacross public clouds (AWS & GCE) and private clouds (Openstack + VMWare)
  • 3.
    VMWARE & OPENSTACKCO-OPETITION Co-operation 1. Openstack using VMWare 2. Complement features - VMWare (live migration, DRS etc) & OpenStack (Object Storage for backups) Competition 1. Customers see Openstack as an alternative to VMWare 2. Migrate VMWare environment to a neutral environment
  • 4.
    CURRENT VMWARE +OPENSTACK INTEGRATION Grizzly 1. ESX Driver (no vMotion, DRS, HA) , Driver runs in 1 nova instance per ESXi host 2. VC Driver (vMotion, DRS, HA), Driver runs in 1 nova instance per ESXi cluster, abstracts cluster Havana 1. “Multi-cluster” management, 1 nova instance for multiple ESXi clusters 2. Block storage volumes (Cinder) can be stored as VMDKs as VMFS datastores (Snapshots go to vCenter datastore). 3. VOVA (VMWare Openstack Virtual Appliance) – packaged OpenStack vApp Icehouse roadmap 1. Cinder replication across different OpenStack installations 2. VMware Datastore Storage Backend (use image location reference instead of migrating the images) 3. VM diagnostics
  • 5.
    HAVANA VMWARE VCDRIVER GlanceStore Glance Images Glance Service Nova VMs Nova Service Cinder Service VCDriver upload_image/ fetch_image Swift Service Cinder Volumes Attach/Detach Vols Swift Store VCenter ESXi Cluster ESXi Cluster Shared VMFS Datastore
  • 6.
    VMWARE BACKUP &DISASTER RECOVERY - TODAY 1. Consolidate backup VMDK disks are snapshotted and mounted to backup proxy server and copies made to backup storage. 2. VMs as physical systems Backup agent runs inside the VM and backs up data to backup servers or attach iSCSI tape as media & copy 3. VMs as a file on ESXi Backup agent runs on the ESXi console & backups the VMDKs as a file to backup server.
  • 7.
    RELIABILITY & HA? Twitter blames two-hour failure on dual data-center crashes. Rawashdeh, VP, Infrastructure Operations Engineering at Twitter. called an "infrastructural double whammy.“ Earlier: • managed hosting services from NTT America • Amazon’s cloud computing services to store and deploy images, including profile pictures The flooded lobby at the Verizon site at 140 West Street during the height of the storm surge Monday night. (Photo:Verizon) Now: Twitter has setup datacenters to cater to its IT needs
  • 8.
    WHAT DOES THISIMPLY ? 1. Distributed environment (not all eggs in one basket) 2. Reliable Backup technology 3. Diverse clouds & infrastructure for backups
  • 9.
    NEED FOR OBJECTSTORE BACKUP Object Store [1] - Distributed data (3 sites) compared to RAID backups (3 copies + Tape backup) 1. Capacity: 60% more efficient 2. Cost : 80 – 90 % cost saved 3. Reliability : 100 Million times more reliable Swift:  Scalable  Durable  In house deployments [no-vendor lockin, cost effective, less latency] [1] Ref: http://www.youtube.com/watch?v=koJwVEDTeg0
  • 10.
    PROPOSED SOLUTION VMWare VCDriver+ Object Store for backup 1. Backup vmdks (root, data & snapshots) on to Object Store (Swift/Amazon S3) 2. Restore vmdks to VMWare shared datastore
  • 11.
    VMWARE VCDRIVER +OBJECT STORE FOR BACKUP Swift Store Glance Store Glance Images Swift Service Glance Service Cinder Volumes Nova VMs Nova Service Cinder Service Attach/Detach Vols Google storage upload_image/ fetch_image S3 Backup/restore images + volumes VCDriver VCenter ESXi Cluster ESXi Cluster Shared VMFS Datastore
  • 12.
    VMWARE VCDRIVER +OBJECT STORE FOR BACKUP (CONTD.,) (root disk, data disk, snapshot disk) Restore Backup VCDriver Swift Service S3 Google Storage Distributed (multi-site), reliable & HA storage vCloudSync (Backup As A Service) Glance Store Shared VMFS Datastore
  • 13.
    VCLOUDSYNC vCloudSync  Backup VMDKto AWS S3 & Google storage (Offline backups)  Restore VMDK back to VMWare  Google agent to migrate VMs between VMWare & GCE (Centos & Ubuntu) Roadmap  Snapshot backup  Integration with VCDriver/OpenStack  AWS agent to migrate VMs between VMWare & AWS (Ubuntu)
  • 14.
    WHY VCLOUDSYNC ? +Agentless backups + Optmized for cloud backups and no local backups + Optmized Chunk size + Multi-part backups + Simultaneous backups + Deduplication + Compression (Ratio - 10:8.9) + ~ 20% better upload performance compared to manual uploads + Agnostic to virtualization or cloud technologies - Backup As A Service Engine + Reliability – meta data based backups
  • 15.
    THANKS ! @vinothiniraju TROV ITConsulting & Services Pvt Ltd, #532, 2nd Floor, AECS Layout, B Block, Bangalore - 560037, India http://trov.co.in Telephone:+91-80-40990162, +91-80-28540594 E-mail: info@trov.co.in

Editor's Notes

  • #3 http://www.slideshare.net/mirantis/how-to-compare-vmware-and-openstack
  • #5 Distributed Resource Scheduler -Dynamic Load Balancing and Resource Allocation for Virtual Machines (Resource pool management & vMotion), HA – server failovers and initiate VMs in the resource pool upon failures.
  • #7 VM as physical systems - + app aware backupsCannot back up VM as a whole
  • #8 Need raw processing power. Clouds – more latencyhttp://www.datacenterknowledge.com/archives/2010/04/16/twiter-will-get-its-own-data-center/http://www.datacenterknowledge.com/archives/2011/03/21/twitter-completes-data-center-expansion/