Island: Local Storage Volume for Cinder


Published on

Presented in Design track of OpenStack Summit, 2012 Fall San Diego.

Published in: Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Snapshot & backend Clay, John Backup API Plugin(driver) + Backend
  • Red is Shortcoming Green is Advantage Island Advantages High speed and low latency in local disk I/O Incremental snapshot Snapshots are stored in objectstorage The time to restore the volume is the same for all snapshots Cost reduction from using inexpensive local disks Raid 10 in local storage have a better reliability Its features are like AWS EBS volumes
  • Design Idea= Local Storage + QCOW2 Image + Dependent Snapshot + Swift
  • When user create a snapshot of volume, island will (1) Create snapshot in image; (2) Upload snapshot to swift
  • Island module
  • IslandDriver Process Attach volume Process (1) Island create the image of the volume. (2) if the volume is new volume, goto (5); (3) if the volume is new volume from snapshot, download the snapshot and write it to the image, goto (5); (4) if the volume is old volume, download the last snapshot and write it to the image; (5) update volume['host'] to new host which the instance besides. the volume is attached to the instance.
  • Create snapshot (1) Island use qemu monitor command to create snapshot in image; (2) Island upload incremental snapshot to swift; (3) Island delete old snapshot in image. NOTE: snapshot can be created when the volume state is attached. When the volume is detached, there are not changes in volume, so it don't need to create snapshot.
  • Point-in-time snapshot (instant snapshot) (即时快照)
  • Except add some code like other plugin do, we did not modify any code in Cinder and Nova.
  • Island: Local Storage Volume for Cinder

    1. 1. Island: Local Storage plugin for Cinder 在这里写上你的标题 Presented at OpenStack Summit, San Diego 副标题文字副标题文字 Hui Cheng/RongZhe Zhu | 作者名字 / 日 2012-09-22 期
    2. 2. Content 1. Why 00 写上你的文字你的文字 2. Design 01 3. Snapshot 02 4. Discussions 03 04 05
    3. 3. Why AWS EBS 写上你的文字你的文字 Island 00Network bottleneck High durability 01 Re-mirroring storm = Incremental snapshot 02 The high performance of Low performance Snapshots store in S3 12 disks RAID10 03 Incremental snapshot Our Environment 04 Snapshots store in swift 05 写上你的文字你的文字 Every x86 server have 1000Mb Network 12 disks
    4. 4. Design – Island in Cinder KEY REST Users AMQP Nova API Volume API SQL POSIX = Nova cinder Cinder RPC RPC DB Nova DB Swift API Compute Volume Snapshot in Swift Manager Manager Host Island KVM Swift Storage Guest imagesIsland Design Idea= Local Storage + QCOW2 Image + Dependent Snapshot + Swift
    5. 5. Design – Incremental Snapshot in Swift 00 VM VM VM Snapshots are saved incrementally. Each subsequent snapshot will only 01 save modified data block to swift. A 02 image image image = Host table of contents points to the latest versions of each data block. Each data block have a reference counter. 03 04 05 snap snap snap 1 2 3 Swift1 2 3 4 1 2 2 3
    6. 6. create_volume delete_volume Design – Island Architecture create_volume_from_snapshot create_snapshot delete_snapshotcreate_vdisk 写上你的文字你的文字 attach_volume 00delete_vdisk detach_volumecreate_snapshot IslandDriver …… 01delete_snapshotread_vdisk 02write_vdisk = get_snapshot put_snapshotget_update_blocks Snapshot 03read_vdisk_meta delete_snapshot get_object 04 put_object delete_object 05 set_object_meta Vdisk ObjectStorage get_object_meta QCOW2 VDMK Swift NFS Glusterfs
    7. 7. Design - Process 00 Create volume Do nothing 01 Create volume from snapshot 02 Do nothing = 03 Delete volume Do nothing 04 Delete snapshot 05 Island delete snapshot in swift
    8. 8. Design - Process Create the image of the volume 写上你的文字你的文字Attach volume 00 01 volume old Check the new volume form snapshot 02 = volume 03 Download the last Download snapshot and snapshot and 04wirte it to the new volume wirte it to the image image 05 Update volume[‘host’]
    9. 9. Design - Process Create snapshot 00 Detach volume 01 Create a new snapshot DB item, set Create snapshot in image snapshot[‘volume_id’] as the volume 02 03 Upload incremental Create snapshot in image snapshot to swift 04 05 Delete old snapshot in Upload incremental snapshot to swfit image Delete the image of the volume
    10. 10. Snapshot – How to Create/Read Snapshot in ImageCreate internal snapshot in imageBecause there写上你的文字你的文字 command to create internal snapshot is not a qemu monitor 00in image when the image was attached. So, we add a new qemu monitorcommand to create point-in-time snapshot in image. 01Accessing QCOW2 image in python 02 =QCOW2 image format use 2-level Lookup Table 03We implement qcow2 python library to read/write qcow2 image data andmetadata. 04 05
    11. 11. Snapshot – How to Implement Incremental Snapshot 00 写上你的文字你的文字 Compare current snapshot pointer table and previous snapshot pointer 01 table in imageWe need to find out updated data blocksbetween two02snapshots. = Find updated data blocks 03 04 Upload updated data blocks to 05 objectstorageFuture: modify qcow2 driver in qemu, markupdated data blocks in pointer table of Upload snapshot pointer table tosnapshot in image. objectstorage
    12. 12. Discussions The 00 写上你的文字你的文字 storage capacity of a x86 server is limit, but if many instances in a server and many volume are attached to 01 these instances, it will be a issue. The 02 = attach volume action is slower than other plugin. 03 Optimization? 04 Make Island as a independent system? 05