Intercloud object storage: colony- Trial to use swift     2012.4.18                                                     @j...
Cloud projects @ NII                       1
Goal: academic community cloud                                  NII CloudAcademicCommunity Cloud                          ...
Colony and dodai in a scenarioVMI: Virtual Machine ImageVM: Virtual Machines            VMI        colony    VMI          ...
Colony         4
Intercloud object storage service                                                               Colony is a project which ...
Users’ points of view   Cloud Services                                                     Cloud-B                        ...
Colony realizes the federation                           Shibboleth IdP                  Authenticate with                ...
Developed software components in colony• Colony-Horizon – based on diablo/stable Horizon with some enhancements    •   Mul...
Colony-horizonUsers can choose swift   Swift -I                         Swift -A                                    9
Colony – keystone               Shibboleth IdP                                        Modifications to keystone           ...
Colony-dispatcher1. Swift client can send requests to Swift-A and Swift-I through Swift Dispatcher2. Swift Dispatcher merg...
CachingSwift Dispatcher can use cache proxy (like squid) perswift proxy to retrieve objects from remote swifts.  A:contain...
Download performance with cache                                     Download performance (5times, average)                ...
Performance : original swift                                          Swift cannot perform well when                      ...
Performance : with cache                 Swift                       Cache works                 @Sapporo                 ...
Performance : with zone-awareness                                            Swift should be zone-aware                   ...
Project status of colony • Prototyping is done in NII research cloud  https://github.com/nii-cloud/colony/ • Future plan  ...
Dodai        18
CaaS : Cluster as a Service                          Clusters for each private cloud   Elastic                   Private c...
Dodai = dodai-deploy + dodai-compute                  Dynamically and securely separated clustersPaaS   Hadoop   MPI      ...
Dodai-deploy• It is a framework for deployment tool   – New software can be added by anyone   – Now supports        • Open...
How to use dodai-deploy• Install   – Just one command to install all-in-one      • curl https://raw.github.com/nii-cloud/d...
What’s inside in dodai-deploy                                        MCollective                  dodai-deploy server   pu...
dodai-compute• In a word   –Bare metal version nova• Futures   –Provide the same interface as nova to    operate on physic...
What’s inside in dodai-compute                                 OpenFlowmodified nova                    controller        ...
Project status of dodai 1. Dodai-deploy    - One of the OpenStack compute automated installations tools    http://docs.ope...
NII research cloud       gunnii                     27
NII academic cloud : gunnii“Elastic Private Cloud”                                on     on     on                        ...
gunnii prototype system                                                                     NII shibboleth IdP  Researcher...
Demo ?    gunnii is shibboleth ready.                                  30
https://github.com/nii-cloud/               Thank you                                31
Upcoming SlideShare
Loading in …5
×

OpenStack design summit (colony session)

1,379
-1

Published on

The presentation at OpenStack design summit (colony session)
on 18th of April, 2012.
http://www.openstack.org/conference/san-francisco-2012/sessions/

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,379
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
37
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

OpenStack design summit (colony session)

  1. 1. Intercloud object storage: colony- Trial to use swift 2012.4.18 @jxtain wide area network - Shigetoshi Yokoyama National Institute of Informatics
  2. 2. Cloud projects @ NII 1
  3. 3. Goal: academic community cloud NII CloudAcademicCommunity Cloud Education Cloud Univ.-X Cloud Univ. -A Cloud Univ.-B Cloud Research Cloud ・・・ Intercloud services Science Information Network 2
  4. 4. Colony and dodai in a scenarioVMI: Virtual Machine ImageVM: Virtual Machines VMI colony VMI Launch machine imagesVirtualMachines VM VM VM VM VM VM VM VM IaaS IaaS IaaS IaaS Cluster Cluster Cluster Cluster dodaiPhysicalMachines SINET (L2VPN) 3
  5. 5. Colony 4
  6. 6. Intercloud object storage service Colony is a project which Nova federates cloud object storage services, like swift Glance Swift for intercloud Swift to realize intercloud object storage service. use Swift for intercloud use Swift forNova Swift for Swift local use intercloud Glance use Glance Swift Nova Swift for intercloud use 5
  7. 7. Users’ points of view Cloud Services Cloud-B Object B1-1 Cloud-A Container B1 Container B2 Container B3 Object B1-2 Object B1-3 Object A1-1 Swift-B Object I1-1 Container A1 Object A1-2 Inter-cloud Container I1 Object I1-2 Container A2 Object A1-3 Inter-cloud Container I8 Object I1-3 Swift-A Container A3 Inter-cloud Container I10 Inter-cloud Container I1 Object I4-1 Inter-cloud Container I4 Object I4-2 Inter-cloud Container I13 Object I4-3 Geographically Inter-cloud Container Object I1-1 Object I1-2 Distributed I1 Inter-cloud Container I2 Object I1-3 Swift-I Inter-cloud Container I3 Inter-cloud Container I4 Object I4-1 Object I4-2 Object I4-3 Inter-cloud object storage service : colony 6
  8. 8. Colony realizes the federation Shibboleth IdP Authenticate with Shibboleth IdP Cloud-A User Colony Apache mod_wsgi mod_shib Colony-horizon Colony-keystone Colony-dispatcher Squid Provide seamless access to Slapd Ubuntu multiple swifts Swift Swift Swift-I Colony-Keystone Colony-Keystone Swift-A Slapd Slapd 7
  9. 9. Developed software components in colony• Colony-Horizon – based on diablo/stable Horizon with some enhancements • Multi-region support – Users can choose which swift is used to store/retrieve objects • Swift Container’s ACL ,metadata support • Swift Object’s metadata support • >5G segment upload support …• Colony-Keystone – based on diablo/stable Keystone with some enhancements • Authenticate with Shibboleth • %{tanant_name} can be used for endpointTemplates in addition to %{tenant_id} for federating cloud services• Colony-Dispatcher - new • Relay requests to multiple object services (and merge response for clients) • Relay requests to a specific object service indicated by URI • Choose the “nearest” swift-proxy server to relay requests • Copy objects among different swifts• Utilities - new • Tools to make admin task easier to provide the federation of object storage services 8
  10. 10. Colony-horizonUsers can choose swift Swift -I Swift -A 9
  11. 11. Colony – keystone Shibboleth IdP Modifications to keystone • Add ePPN field to keystone schema • ADD rest api services to create token by ePPN (/token_by/eppn) and email address(/token_by/email) (/ /email) • Add a rest api service to register/update ePPN (/users/{user_id}/eppn) (/users/{1. ID/passwd 2. Attribute: ePPN, mail_addr 0-1. User registration by mail_addr 0-2 . Associate ePPN to mail_addr by initial access Shibboleth SP Colony- Colony-Horizon 3. Attribute: ePPN Colony- 4. auth_token Keystone 10
  12. 12. Colony-dispatcher1. Swift client can send requests to Swift-A and Swift-I through Swift Dispatcher2. Swift Dispatcher merges the response from each Swift and send it to Swift Client Swift Client Requests modified for merging responses. A:container1 •Account Info A:container2 •Container List •X-Copy-from/to I:container1 I:container2 Colony DispatcherResponse merged by Swift Proxy Swift Proxy Swift ProxyColony Dispatcher hasa prefix to indicatewhich Swift is used tostore. Swift-A (local) Swift-I (intercloud ) 11
  13. 13. CachingSwift Dispatcher can use cache proxy (like squid) perswift proxy to retrieve objects from remote swifts. A:container1 A:container2 I:container1 Colony Dispatcher I:container2 Cache(Proxy) Swift Client Swift Proxy Swift Proxy Swift Proxy Swift-A (local) Swift-I (intercloud ) 12
  14. 14. Download performance with cache Download performance (5times, average) 300,000,000 250,000,000 200,000,000 Cache is bytes/sec 150,000,000 effective, 100,000,000 especially for small size 50,000,000 0 objects. 1K(bytes) 1M(bytes) 10M(bytes) 100M(bytes) 1G(bytes) 5G(bytes) Normal 13,201 11843208 77113446 152263382 163115324 154252275 Cache 382,004 135938185 226800796 256492362 261189612 260956927・AMD Opteron(tm) Processor 6128 2000Mhz 16core Ubuntu Server 11.04 x86_64 Client・32GB Memory Colony-Dispatcher・10000baseT 10000Mb/s Full Duplex Squid Swift-Proxy・AMD Opteron(tm) Processor 6128 2000Mhz 16core Ubuntu Server 11.04 x86_64 Swift-Storage(account, container, object)x 3・32GB Memory・10000baseT 10000Mb/s Full Duplex 13
  15. 15. Performance : original swift Swift cannot perform well when confgirure it in inter-region. Swift @Sapporo •Upload – Always effected by worst case 1 2 3 4 5 1K 1,755 2,165 395 3,178 1.375 1M 431,840 446,824 537,768 412,542 43,307 10M 710,179 741,350 737,565 723,446 731,824 100M 780,548 775,131 788,620 782,503 786,224 Swift •Download – worse case is ramdomly choosed @Tokyo Swift@ (about 1/3) Chiba Client 1 2 3 4 5 6.15Mbits/sec 1K 11,608 13,293 12,916 14,691 12,126 1G 6.29Mbits/sec 7.80Mbits/sec 1M 3,427,827 3,427,402 3,455,788 673,614 3,386,009 7.73Mbits/sec netperf 10M 958,018 959,098 7,188,901 974,072 949,389 941.23Mbits/sec10G 100M 1,109,440 98,400,555 99,641,184 15,411,221 92,400,774 929.15Mbits/sec 14
  16. 16. Performance : with cache Swift Cache works @Sapporo •Upload – Always effected by worst case 1 2 3 4 5 1K 2,388 2,027 1,629 2,718 2,423 1M 521,456 479,606 488,142 455,359 541,622 10M 727,240 678,511 724,562 730,127 699,518 100M 786,893 777,227 787,707 782,546 784,544 Swift @Tokyo Swift@ Chiba •Download (through Cache) 1 2 3 4 5 Client 6.15Mbits/sec 1K 398,133 415,753 397,979 446,576 406,510 1G 6.29Mbits/sec 1M 9,238,473 202,193,598 177,694,628 161,992,275 134,450,057 7.80Mbits/sec 7.73Mbits/sec netperf 10M 55,186,177 288,776,403 319,892,614 331,691,392 307,500,293 941.23Mbits/sec 100M 473,135,008 471,484,134 467,654,981 412,153,419 451,464,516 10G 929.15Mbits/sec 15
  17. 17. Performance : with zone-awareness Swift should be zone-aware for geographically distributed use. Swift Modifications for zone-awareness (just a hack) @Sapporo •Uploading • Calculate the number of unfinished tasks in the send queue for each area and when one area are much worth than others, quit the uploading job for its area. •Downloading • Check connection performance of object servers and try to retrieve an object from the fastest one. Swift •Upload @Tokyo Swift@ Chiba 1 2 3 4 5 1K 11,356 13,157 13,074 12,758 12,680 Client 6.15Mbits/sec 1M 9,824,750 11,205,249 7,599,312 10,931,206 11,199,982 1G 6.29Mbits/sec 7.80Mbits/sec 10M 52,294,403 51,437,092 51,050,686 52,641,471 52,300,141 7.73Mbits/sec netperf 100M 97,937,987 101,847,002 102,385,002 102,413,801 101,462,855 941.23Mbits/sec 10G 929.15Mbits/sec 16
  18. 18. Project status of colony • Prototyping is done in NII research cloud https://github.com/nii-cloud/colony/ • Future plan - Performance enhancements - Real deployment and evaluation 17
  19. 19. Dodai 18
  20. 20. CaaS : Cluster as a Service Clusters for each private cloud Elastic Private cloud - A Private cloud - B Private cloud - C Private Cloud Release servers to server resource pool Allocate servers from server resource pool Server resource pool 19
  21. 21. Dodai = dodai-deploy + dodai-compute Dynamically and securely separated clustersPaaS Hadoop MPI Hadoop MPI Hadoop MPI … SunGrid Hadoop EngineIaaS OpenStack-2 OpenStack-1 Eucalyptus Upper layer: Deploying software [dodai-deploy]CaaS Lower layer: Preparation of nodes [dodai-compute] 20
  22. 22. Dodai-deploy• It is a framework for deployment tool – New software can be added by anyone – Now supports • Openstack Essex • Openstack Diablo • Hadoop • SGE• It is fast – Do installation parallel on different machines.• It is easy to use – Not just Web UI – CLI for maintainer – Rest API for integration 21
  23. 23. How to use dodai-deploy• Install – Just one command to install all-in-one • curl https://raw.github.com/nii-cloud/dodai-deploy/master/setup-env/setup- all-in-one.sh | sudo sh• Use – Web UI • Learn step-by-step guidance at http://$dodai_deploy_server:3000/ – REST APIs • Try at http://$dodai_deploy_server:3000/rest_apis/index.html – CLI • script/cli.rb list SERVER 22
  24. 24. What’s inside in dodai-deploy MCollective dodai-deploy server puppet Ruby on Rails dodai-deploy nodes 23
  25. 25. dodai-compute• In a word –Bare metal version nova• Futures –Provide the same interface as nova to operate on physical machines –Separate network segments with Open Flow Controller 24
  26. 26. What’s inside in dodai-compute OpenFlowmodified nova controller cobbler OpenFlow switch 25
  27. 27. Project status of dodai 1. Dodai-deploy - One of the OpenStack compute automated installations tools http://docs.openstack.org/trunk/openstack-compute/admin/content/openstack-compute-deployment-tool-with-puppet.html https://github.com/nii-cloud/dodai-deploy/ - Used in NII research cloud development 2. Dodai-compute https://github.com/nii-cloud/dodai-compute/ - Used in NII research cloud development 3. Future plan - Enahance based on Feedbacks from NII researchers - Experiments on intercloud compute service 26
  28. 28. NII research cloud gunnii 27
  29. 29. NII academic cloud : gunnii“Elastic Private Cloud” on on on demand demand demand Cluster as a Service Based on Deployment Tool dodai-deploy generalized Compute dodai-compute bare metal supported 28
  30. 30. gunnii prototype system NII shibboleth IdP Researchers L3 switch Object storage (Swift) GUI server OpenFlow Switch/controller Storage nodes X 8 (initial) NFS Deployed Hadoop, OpenStack, Sun Grid Engine Compute nodes×40 (initial) 29
  31. 31. Demo ? gunnii is shibboleth ready. 30
  32. 32. https://github.com/nii-cloud/ Thank you 31
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×