OCDET Activity and Glusterfs


Published on

Slides for LinuxCon / Open Cloud Japan 2013

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

OCDET Activity and Glusterfs

  1. 1. OCDET Activity and GlusterfsMay 31, 2013Gluster Workshop @ LinuxCon / OpenCloud Japan 2013Copyright (c) OCDET All Rights Reserved.May 31, 2013OCDET / NTT DATA CORPORATIONMasanori Itoh
  2. 2. Who I amMasanori Itoh– Professional Affiliation• NTT DATA CORPORATION– A Japanese System Integrator (NOT a Telco ! ☺)– Activities• OCDET (Steering Committee) Today, I’m on behalf ofCopyright (c) OCDET All Rights Reserved. 1• OCDET (Steering Committee)• Japan OpenStack Users Group• etc.– Speciality• Operating System– Filesystem, Networking, …• Trouble Shooting … :oToday, I’m on behalf ofOCDET !
  3. 3. Agenda1. What is OCDET ?2. OSS IaaS with Glusterfs3. Future Expectations and Ideas for Glusterfs4. SummaryCopyright (c) OCDET All Rights Reserved. 2
  4. 4. 1. What is OCDET ?1. What is OCDET ?3
  5. 5. What is OCDET?Home Page : http://ocdet.org/ (Sorry, No English page… )Copyright (c) OCDET All Rights Reserved. 4
  6. 6. What is OCDET?OCDET stands for :Open Cloud Demonstration Experiment Taskforceオープン クラウド 実証 実験 タスクフォースBTW, “Demonstration Experiment” ?Copyright (c) OCDET All Rights Reserved. 5BTW, “Demonstration Experiment” ?– In Japanese,• “実証実験” (Jissho-Jikken)– In Short,• An Activity on POC : Proof-of-Concept (of Something)
  7. 7. What is OCDET? : the Goalthe GoalTo Promote “Open Source Cloud Computing Components”• From the Lowest Layer to Upper Layer things.• Even including Hardware (!?)Prove that we can Build Mature Infrastructure Using ThemBy doing various POC activitiesCopyright (c) OCDET All Rights Reserved. 6By doing various POC activitiesUnder Fully Volunteered and Non-Profit OrganizationAnd Share the Results Publicly• Documentation– Installation/Configuration Procedures, System DesignGuidelines, Knowhows , … etc.• Tools, Sample Configuration Files, …
  8. 8. What is OCDET? : OverviewFormed on:Feb. 3, 2012Participants:24 Organization MembersFounding Organizations : Hokkaido University,NTT Communications, Cloud Business Association, CUPACopyright (c) OCDET All Rights Reserved. 7NTT Communications, Cloud Business Association, CUPA68 Active Volunteer Members (May 2013)OrganizationSteering CommitteeStudy Groups / Working Groups
  9. 9. What is OCDET? : Press CoverageStudy Groups / Working Groupshttp://ocdet.org/information.html#pageLink02Copyright (c) OCDET All Rights Reserved. 8
  10. 10. What is OCDET? : OrganizationStudy Groups / Working Groupshttp://ocdet.org/information.html#pageLink02Copyright (c) OCDET All Rights Reserved. 9
  11. 11. What is OCDET? : OrganizationStudy Groups / Working GroupsOpen IaaS Study Group• OpenStack WG• CloudStack WG• System Management WGOpen Storage Study GroupOrchestration SIGCopyright (c) OCDET All Rights Reserved. 10Open Storage Study Group• Swift, Glusterfs, etc.Open Network Study GroupOpen PaaS Study Group• CloudFoundry <--> Japan CloudFoundry User GroupOpen Facility Study Group• OpenCompute WG <--> OCP Japan
  12. 12. 2. IaaS Software and Glusterfs2. IaaS Software and Glusterfs11
  13. 13. IaaS Software and Glusterfs : IaaS at a GlanceProvide Infrastructure Level Resources on DemandCompute ResourceCPU, Memory, …Network ResourceL2 Network, IP address Management, Router, Firewall, LoadBalancer, VPN, …Copyright (c) OCDET All Rights Reserved. 12Balancer, VPN, …Storage ResourceFile Access StoreBlock Access StoreObject Storage Store
  14. 14. IaaS Software and Glusterfs : ExamplesExamples of Public IaaS ServiceAmazon Web ServicesRackSpace CloudHP Cloud Servicesetc.Copyright (c) OCDET All Rights Reserved. 13Examples of Open Source Software for IaaSOpenStackCloudStackEucalyptusOpenNebulaetc.
  15. 15. IaaS Software and Glusterfs : Typical Use CasesVM Images Store (Not always Expected being Persistent)Image Templates and Image Instances of Running VMsNeed to be Either of File or Block Access I/FExternal(/Additional) VolumesTypically, Expected as ‘Persistent Storage’ for Databaseand to have Block Storage I/F (to the guest OS instances)Copyright (c) OCDET All Rights Reserved. 14and to have Block Storage I/F (to the guest OS instances)VM Image/External Volumes Snapshot Store(Tenant) Backup / Bulk Data StoreXXX as a backend of HDFS (gluster already has!)Object Storage I/F Fits for this Purpose Best. (Swift API?)etc.
  16. 16. Test Case 1 : OpenStack Configuration ExampleIntranetInternet /IntranetZone #1External NetworkEnd User PCFirewallAmazon EC2 type Network ModelNW Gateway Node aggregatestenant external traffic(*1) Cloud ControllerAPI, Scheduler, RabbitMQ, RDBMS etc.Courtesy of Internet Initiative Japan Inc.Copyright (c) OCDET All Rights Reserved.Quantumagent(L2/L3)CloudControllerGlance(Swift)VMImageNova ComputeQuantum AgentVMVM…Internal Network (VLAN/GRE etc.)Management/Storage NetworkDB(MySQL)Nova ComputeQuantum AgentVMVMovs ovsManagement/Storage NetworkNW GatewayHypervisorsgluserglusergluster. . .gluster
  17. 17. Test Case 1 : VM Image Store w/OpenStackShared FS byGluster for VMImagesvolume: gvol1VMVMVMVMVM Service NetworkOpenStack Folsom + CentOS 6.3 (x86_64) + gluster 3.3glusterCopyright (c) OCDET All Rights Reserved.Management/mnt/sdb1 /mnt/sdb1 /mnt/sdb1volume: gvol1/mnt/gluster1 /mnt/gluster1 /mnt/gluster1NFS mountbricks
  18. 18. Test Case 2 : CloudStack Configuration ExampleIntranetInternet /IntranetZone/PodExternal NetworkEnd User PCTraffic forVirtual RoutersDNAT based Network ModelPer-tenant Virtual Router VM willbe created and it proxies tenanttraffic.Courtesy of DataHotel Co., Ltd.for SecondaryStoageCopyright (c) OCDET All Rights Reserved.Mgmt.Server.NFSVMImageComputeVM…DBComputeVMComputeVirtualRouterComputeVirtualRoutervnic0vnic1VMvnic0vnic1SSVMCPVMManagement/Storage NetworkManagement/Storage Networkgluster
  19. 19. Test Case 2 : VM Image Store w/CloudStackShared FS byGluster for VMImagesVMVMVMVMVM Service NetworkCloudStack 4.0 + CentOS 6.4(x86_64) + gluster 3.3 (4nodes)Copyright (c) OCDET All Rights Reserved.Management / Storage Traffic/mnt/sdb1 /mnt/sdb1 /mnt/sdb1volume: gvol1 /mnt/gluster1 /mnt/gluster1 /mnt/gluster1NFS mountbrick. . .
  20. 20. Summary of our (current) POC ExperiencesVM Image StoreGlusterfs is QUITE EASY to install/configure and WORKSFAIRLY WELLWith both OpenStack and CloudStackWithout almost No Pitfalls except trivial configuration mistakesForgot to add iptables ACCEPT rule for NFS etc. :oCopyright (c) OCDET All Rights Reserved. 19Forgot to add iptables ACCEPT rule for NFS etc. :oBandwidth Bottleneck when Many VMs launched at onceetc.Object StorageTBDVolume BackendTBD
  21. 21. OCDET (Possible) Future PlanLarge Scale Evaluation Using STARBEDReserved 140 nodes (Now, Swift evaluation is ongoing)Gluster Swift API EvaluationOpenStack BlockStorage (Cinder) driver EvaluationInfiniBand Support Evaluation (?)Copyright (c) OCDET All Rights Reserved. 20InfiniBand Support Evaluation (?)etc.
  22. 22. 3. Future Expectations and (Stupid?) Ideas3. Future Expectations and (Stupid?) Ideasfor Glusterfs21
  23. 23. Future Expectations and (Stupid?) IdeasEnhancements of OpenStack Cinder Backend DriverAdd Volume Snapshot FeatureSnapshot Support (versioning?)Some Handoffs with Local Filesystems? (e.g. btrfs?)libvirt support of qemu-gluster integrationCopyright (c) OCDET All Rights Reserved. 22libvirt support of qemu-gluster integrationImprovement of InfiniBand Support QualityI saw a gluster-rdma system hung up under heavy I/Oload (using iozone) using gluster 3.3beta2 formerly…Linux Upstream Code Merge(?)
  24. 24. Future Expectations and (Stupid?) IdeasErasure Coding Support (Like the case of HDFS) ?To Reduce Required Physical Capacity without LosingRedundancy LevelTo Reduce Storage I/O TrafficReplica Locality Control ??Copyright (c) OCDET All Rights Reserved. 23Replica Locality Control ??To Reduce Replication TrafficNFS/RDMA on top of Glustefs ?Geologically Distributed “Synchronous” Replication?For DR. In this sense, Single Master makes sense …
  25. 25. Natural Expectations…http://office.microsoft.com/en-us/images/similar.aspx#ai:MP900442518|Copyright (c) OCDET All Rights Reserved. 24
  26. 26. 4. Summary25
  27. 27. Conclusion and SummaryGlusterfs is a Fairly Good Solution for IaaSDeployments ☺OCDET is Further Evaluating Glusterfsand Sharing our Results PubliclyJoin us ! and Let’s Play with Glusterfs together !☺Copyright (c) OCDET All Rights Reserved. 26Join us ! and Let’s Play with Glusterfs together !☺
  28. 28. Thank You !Copyright (c) OCDET All Rights Reserved. 27Thank You !