Downloaded 14 times
![20
Swift shared cluster: ex)
Swift-Proxy
[filter:keystone]
reseller_prefix = nc_
[filter:ceilometer]
reseller_prefix = zc_
[filter:keystone]
reseller_prefix = gac_
[filter:keystone]
reseller_prefix =
[filter:keystone]
## reseller_prefix =
[filter:keystone]
reseller_prefix = zc_
[filter:ceilometer]
reseller_prefix = zc_
Swift-Proxy
[filter:keystone]
reseller_prefix = nc_
[filter:ceilometer]
reseller_prefix = zc_
[filter:keystone]
reseller_prefix = gac_
[filter:keystone]
reseller_prefix =
[filter:keystone]
## reseller_prefix =
[filter:keystone]
reseller_prefix = zc_
[filter:ceilometer]
reseller_prefix = zc_
<< account-server
<< container-server
<< object-server
( 2 x 5 nodes )
reverse-proxyreverse-proxyLVS-DSR LVS-DSR
keystone
ConoHa VPS cluster (Grizzly)
https://swift-url/<tenant-id>/
keystone
AppsCloud cluster (Havana)
https://swift-url/AUTH_<tenant-id>/
keystone
AppsCloud cluster (Juno)
https://gac-swift-url/gac_<tenant-id>/
keystone
ConoHa Cloud cluster (Juno)
https://nc-swift-url/nc_<tenant-id>/
keystone
Z.com Cloud (Juno)
https://zc-swift-url/zc_<tenant-id>/
the Internet](https://image.slidesharecdn.com/openstack-days-taiwan-2016-0712-03-ipad-160712050841/85/Openstack-days-taiwan-2016-0712-19-320.jpg)
![33
Public API global network
LVS-DSR
(act-stby)
the Cloud
(Internet)
HAProxy
LVS
heatbeat
api-reverse-proxy01 api-reverse-proxy02elvs01
elvs02
VMx2
LVS
heatbeat
VMx2
HAProxy
ext-api-wrapper01
php + httpd
- keystone
- nova
- cinder
- neutron
- glance
- account
ext-api-wrapper02
php + httpd
- keystone
- nova
- cinder
- neutron
- glance
- account
control-nodes01
- keystone API
- nova API
- cinder API
- neutron API
- glance API
control-nodes02
- keystone API
- nova API
- cinder API
- neutron API
- glance API
OpenStack Management network
step 1)
step 2)
step 3)
step 4)
public API: step 1, step 2)
step 1) LVS-DSR (L4) is received https(tcp/443) packet,
then forward api-reverse-proxy real IP’s.
step 2) HAProxy has valid API ACL and backend server configurations.
IF HAProxy allowed POST “/v2.0/tokens”, then the request call to ext-api-wrapper0[12].](https://image.slidesharecdn.com/openstack-days-taiwan-2016-0712-03-ipad-160712050841/85/Openstack-days-taiwan-2016-0712-31-320.jpg)
![34
Public API global network
LVS-DSR
(act-stby)
the Cloud
(Internet)
HAProxy
LVS
heatbeat
api-reverse-proxy01 api-reverse-proxy02elvs01
elvs02
VMx2
LVS
heatbeat
VMx2
HAProxy
ext-api-wrapper01
php + httpd
- keystone
- nova
- cinder
- neutron
- glance
- account
ext-api-wrapper02
php + httpd
- keystone
- nova
- cinder
- neutron
- glance
- account
control-nodes01
- keystone API
- nova API
- cinder API
- neutron API
- glance API
control-nodes02
- keystone API
- nova API
- cinder API
- neutron API
- glance API
OpenStack Management network
step 1)
step 2)
step 3)
step 4)
public API: step 3), step 4)
step 3) ext-api-wrapper0 [12], it is a php program.
request URI and header, and the input value of json of the body was confirmed
by php, and then call the real OpenStack API as the next processing.
step 4) OpenStack API that is checked the input value will be run.](https://image.slidesharecdn.com/openstack-days-taiwan-2016-0712-03-ipad-160712050841/85/Openstack-days-taiwan-2016-0712-32-320.jpg)
Uploaded byNaoto Gohko
Openstack days taiwan 2016 0712
1. GMO Internet operates multiple public cloud services using OpenStack including ConoHa public cloud and GMO AppsCloud. 2. They have a limited number of staff developing and operating OpenStack services across many clusters but must run a large number of OpenStack services. 3. They have upgraded their OpenStack installations over time from Diablo to Juno, expanding services from basic compute to block storage, object storage, load balancing, and more.
More Related Content
![[OpenStack Day in Korea 2015] Track 1-6 - 갈라파고스의 이구아나, 인프라에 오픈소스를 올리다. 그래서 보이...](https://cdn.slidesharecdn.com/ss_thumbnails/16-150213044916-conversion-gate02-thumbnail.jpg?width=640&height=640&fit=bounds)
![[OpenInfra Days Korea 2018] (Track 3) - CephFS with OpenStack Manila based on...](https://cdn.slidesharecdn.com/ss_thumbnails/36openinfra-2018-naver-cephfs-180704055415-thumbnail.jpg?width=640&height=640&fit=bounds)
![[Open infra] how to calculate the cloud system operating rate](https://cdn.slidesharecdn.com/ss_thumbnails/openinfrahowtocalculatethecloudsystemoperatingrate-210911051805-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Openstack days taiwan 2016 0712
![[OpenStack Day in Korea 2015] Track 3-2 - Huawei Cloud Computing Powered by O...](https://cdn.slidesharecdn.com/ss_thumbnails/32-150213065750-conversion-gate02-thumbnail.jpg?width=640&height=640&fit=bounds)
![How Big Brands are Taking Your Traffic in Alberta [Data Inside].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/howbigbrandsaretakingyourtrafficinalbertadatainside-260123180142-42d276f3-thumbnail.jpg?width=640&height=640&fit=bounds)
Openstack days taiwan 2016 0712
- 1. 1 ~ Architecture ofour public clouds ~ OpenStack Days Taiwan Jul 12, 2016 Naoto Gohko (@naoto_gohko) GMO Internet, Inc. How is GMO Ineternet using OpenStack for Public Cloud Slide URL http://www.slideshare.net/chroum/openstack-days-taiwan-2016-0712-public-cloud-arch ConoHa public cloud (lang zh) https://www.conoha.jp/zh/ ConoHa public cloud (lang en) https://www.conoha.jp/en/
- 2. 2 Public Clouds We areoffering multiple public cloud services.
- 3. 3 Physical Servers Running VMPhysicalServer 1508 25294 Created VM Running Infrastructure (2015/10) 137223
- 4.
- 5. 5 Cloud service developmentteam: (abount 30 people) – OpenStack Neutron team: 4 people • Neutron driver / modification / engineering – Cloud API development team: 5 people • Public API validation program • OpenStack modification / scaduler programing / keystone – Cloud Infra. development team: 11 people • Security engineering / glance driver / cinder driver / nova additional extensions / construction of OpenStack infra. – Applicatoin cloud service development team: 5 people • Billing engineering / staff tools / GMO AppsCloud web GUI Additional engineering team: many people (30 ~) – QA Team / Server Engineering Team / GUI development Team – Network Engineering Team / SaaS development Team – CRM backend and billing Team Cloud service development team: Now(2016)
- 6. 6 Cloud service developmentteam: Office(2016) #1 Neutron Team And Cloud API Team Cloud Infra. Team And AppsCloud Team
- 7. 7 Cloud service developmentteam: Office(2016) #2 Neutron Team And Cloud API Team Cloud Infra. Team And AppsCloud Team
- 8. 8 Limied number ofpeople. But, we have to run a lot of OpenStack service clusters.
- 9. 9 Service developmemt history byOpenStack
- 10. 10 Swift cluster GMO Internet,Inc.: VPS and Cloud services Onamae.com VPS (2012/03) : http://www.onamae-server.com/ Forcus: global IPs; provided by simple "nova-network" tenten VPS (2012/12) http://www.tenten.vn/ Share of OSS by Group companies in Vietnam ConoHa VPS (2013/07) : http://www.conoha.jp/ Forcus: Quantam(Neutron) overlay tenant network GMO AppsCloud (2014/04) : http://cloud.gmo.jp/ OpenStack Havana based 1st region Enterprise grade IaaS with block storage, object storage, LBaaS and baremetal compute was provided Onamae.com Cloud (2014/11) http://www.onamae-cloud.com/ Forcus: Low price VM instances, baremetal compute and object storage ConoHa Cloud (2015/05/18) http://www.conoha.jp/ Forcus: ML2 vxlan overlay, LBaaS, block storage, DNSaaS(Designate) and original services by keystone auth OpenStack Diablo on CentOS 6.x Nova Keystone Glance Nova network Shared codes Quantam OpenStack Glizzly on Ubuntu 12.04 Nova Keystone Glance OpenStack Havana on CentOS 6.x Keystone Glance Cinder Swift Swift Shared cluster Shared codes KeystoneGlance Neutron Nova Swift Baremetal compute Nova Ceilometer Baremetal compute Neutron LBaaS ovs + gre tunnel overlay Ceilometer Designate SwiftOpenStack Juno on CentOS 7.x NovaKeystone Glance Cinder Ceilometer Neutron LBaaS GMO AppsCloud (2015/09/27) : http://cloud.gmo.jp/ 2nd region by OpenStack Juno based Enterprise grade IaaS with High IOPS Ironic Compute and Neutron LBaaS Upgrade Juno GSLB Swift Keystone Glance CinderCeilometer Nova Neutron Ironic LBaaS
- 11.
- 12. 12 OpenStack Swift: sharedcluster
- 13. 13 Swift Hardware: Objectnodes • Boot: SSD x2 • HDD: 4TB x12 • E3-1230 v3 @ 3.30GHz • Memory 16GB • 10GbE x2 (SFP+) (Intel NIC) ASUSTeK COMPUTER INC. RS300-H8-PS12
- 14. 14 Hardware: LVS-DSR andreverse-proxy(Layer7) nodes • Boot: SSD x2 • E3-1230 v3 @ 3.30GHz • Memory 16GB • 10GbE NIC x1 (Intel NIC) Supermicro microblade 8 blade nodes type
- 15. 15 Hardware: swift-proxy nodes •Boot: HDD x6 (1.7TB) – Ceilometer Log disk – (Swift all request billing data) • E5620 @ 2.40GHz x2 CPU • Memory 64GB • NIC: 10GbE SFP+ x2(Intel NIC) System x3550 M3 (old IBM) Hardware: account/container-server nodes • Boot: HDD x2 • Account/Container storage: SSD x2 • E5620 @ 2.40GHz x2 CPU • Memory 64GB • NIC: 10GbE SFP+ x2(Intel NIC) System x3550 M3 (old IBM)
- 16. 16 Swift cluster (Havanato Juno upgrade) SSD storage: container/account server at every zone
- 17. 18 swift proxy keystone OpenStack Swiftcluster (5 zones, 3 copy) swift proxy keystone LVS-DSRLVS-DSR HAProxy(SSL)HAProxy(SSL) Xeon E3-1230 3.3GHz Memory 16GB Xeon E3-1230 3.3GHz Memory 16GB Xeon E5620 2.4GHz x 2CPU Memory 64GB swift objects swift objects Xeon E3-1230 3.3GHz swift account swift container Xeon E5620 2.4GHz x 2CPU Memory 64GB, SSD x 2 swift objects swift objects Xeon E3-1230 3.3GHz swift account swift container Xeon E5620 2.4GHz x 2CPU Memory 64GB, SSD x 2 swift objects swift objects Xeon E3-1230 3.3GHz swift account swift container Xeon E5620 2.4GHz x 2CPU Memory 64GB, SSD x 2 swift objects swift objects Xeon E3-1230 3.3GHz swift account swift container Xeon E5620 2.4GHz x 2CPU Memory 64GB, SSD x 2 swift objects swift objects Xeon E3-1230 3.3GHz swift account swift container Xeon E5620 2.4GHz x 2CPU Memory 64GB, SSD x 2
- 18. 19 swift objects swift objects swiftobjects swift objects swift objects swift objects swift objects swift objects swift objects swift objects swift proxy keystone Havana AppsCloud swift proxy keystone Grizzly ConoHa Havana To Juno swift account swift container swift account swift container swift account swift container swift account swift container swift account swift container swift proxy keystone Juno ConoHa swift proxy keystone Juno AppsCloud Swift cluster: multi-auth and multi-endpoint swift proxy keystone Juno Z.com
- 19. 20 Swift shared cluster:ex) Swift-Proxy [filter:keystone] reseller_prefix = nc_ [filter:ceilometer] reseller_prefix = zc_ [filter:keystone] reseller_prefix = gac_ [filter:keystone] reseller_prefix = [filter:keystone] ## reseller_prefix = [filter:keystone] reseller_prefix = zc_ [filter:ceilometer] reseller_prefix = zc_ Swift-Proxy [filter:keystone] reseller_prefix = nc_ [filter:ceilometer] reseller_prefix = zc_ [filter:keystone] reseller_prefix = gac_ [filter:keystone] reseller_prefix = [filter:keystone] ## reseller_prefix = [filter:keystone] reseller_prefix = zc_ [filter:ceilometer] reseller_prefix = zc_ << account-server << container-server << object-server ( 2 x 5 nodes ) reverse-proxyreverse-proxyLVS-DSR LVS-DSR keystone ConoHa VPS cluster (Grizzly) https://swift-url/<tenant-id>/ keystone AppsCloud cluster (Havana) https://swift-url/AUTH_<tenant-id>/ keystone AppsCloud cluster (Juno) https://gac-swift-url/gac_<tenant-id>/ keystone ConoHa Cloud cluster (Juno) https://nc-swift-url/nc_<tenant-id>/ keystone Z.com Cloud (Juno) https://zc-swift-url/zc_<tenant-id>/ the Internet
- 20. 21 OpenStack history of computingenvironment
- 21. 22 Oname.com VPS(Diablo) • ServiceXaaS model: – VPS (KVM, libvirt) • Network: – 1Gbps • Network model: – Flat-VLAN (Nova Network), without floting IP(no L3) – IPv4 only • Public API – None (only web-panel) • Glance – Public image only. OpenStack service: Onamae.com VPS(Diablo)
- 22. 23 ConoHa(Grizzly) • Service XaaSmodel: – VPS + Private networks (KVM + ovs) • Network model: – Flat-VLAN + Quantam ovs-GRE overlay – IPv6/IPv4 dualstack • Network: – 10GE wired(10GBase-T) • Public API: None (only web) • Glance – Only Public image • Cinder: None • ObjectStorage – Swift (After Havana) OpenStack service: ConoHa(Grizzly, 2013/07)
- 23. 24 Grizzly • Quantam Network: –It was using the initial version of the Open vSwitch full mesh GRE-vlan overlay network with LinuxBridge Hybrid But When the scale becomes large, Localization occurs to a specific node of the communication of the GRE-mesh-tunnel (with under cloud network(L2) problems) (Broadcast storm?) OpenStack service: ConoHa(Grizzly)
- 24. 25 • Service XaaSmodel: – KVM compute + Private VLAN networks + Cinder + Swift • Network: – 10Gbps wired(10GBase SFP+) • Network model: – IPv4 Flat-VLAN + Neutron LinuxBridge(not ML2) + Cisco Nexsu L2 sw/port driver – Brocade ADX L4-LBaaS original driver • Public API – Provided the public API • Ceilometer (Billing) • Glance : Provided(GlusterFS) • Cinder : HP 3PAR(Active-Active Multipath original) + NetApp • ObjectStorage : Swift cluster • Bare-Metal Compute – Modifiyed cobbler bare-metal deploy driver – Cisco Nexsus switch bare-metal networking driver (L2 tenant NW) OpenStack service: GMO AppsCloud(Havana)
- 25. 26 OpenStack service: GMOAppsCloud model compute vm NIC Vlan network bridge NIC vlan tap vNIC Vlan network vNIC bridge vlan tap compute NIC bridge NIC vlan bridge vlan public network Neutron LinuxBridge model(very Fast, simple is Best) this cloud is optimized services for the GAME server.
- 26. 27 Cisco Nexsus L2sw/Port manage driver(self made) • L2 resource is limited / SW CPU – MAC ADDRESS – VLAN per Network – VLAN per Port Allowed VLAN to trunked port is allowed only VLAN to be used in LinuxBridge in VM/Baremetal Compute node. – Baremetal : link aggregation port – Port discovery using by lldp • Cisco Nexsus NX-OS – Server: LACP : port-Channel Active-Active link aggreration Fully redundant server (Act-Act link aggreration) Nexus 5k’s (VPC) Nexus 2k: FEX’s (dual homed) Compute node Baremetal Compute node Switch/Port API server Cisco Nexsus Fabric SW Manage NW OpenStack Manage NW
- 27. 29 Nova-baremetal(havana)/Ironic(juno) ansible Baremetal networking •Bonding NIC + lldp discovery • Taged VLAN • allowd VLAN + dhcp native VLAN
- 28.
- 29. 31 Public API securityand load balance: • LVS-DSR • L7 reverse-proxy • API validation wrapper
- 30. 32 public API Web panel(httpd,php) API wrapper proxy (httpd, php Framework: fuel php) Nova API Customer sys API Neutron API Glance API OpenStack API for input validation Customer DB Keystone API OpenStack API Cinder APICeilometer API Endpoint L7:reverse proxy Swift Proxy
- 31. 33 Public API globalnetwork LVS-DSR (act-stby) the Cloud (Internet) HAProxy LVS heatbeat api-reverse-proxy01 api-reverse-proxy02elvs01 elvs02 VMx2 LVS heatbeat VMx2 HAProxy ext-api-wrapper01 php + httpd - keystone - nova - cinder - neutron - glance - account ext-api-wrapper02 php + httpd - keystone - nova - cinder - neutron - glance - account control-nodes01 - keystone API - nova API - cinder API - neutron API - glance API control-nodes02 - keystone API - nova API - cinder API - neutron API - glance API OpenStack Management network step 1) step 2) step 3) step 4) public API: step 1, step 2) step 1) LVS-DSR (L4) is received https(tcp/443) packet, then forward api-reverse-proxy real IP’s. step 2) HAProxy has valid API ACL and backend server configurations. IF HAProxy allowed POST “/v2.0/tokens”, then the request call to ext-api-wrapper0[12].
- 32. 34 Public API globalnetwork LVS-DSR (act-stby) the Cloud (Internet) HAProxy LVS heatbeat api-reverse-proxy01 api-reverse-proxy02elvs01 elvs02 VMx2 LVS heatbeat VMx2 HAProxy ext-api-wrapper01 php + httpd - keystone - nova - cinder - neutron - glance - account ext-api-wrapper02 php + httpd - keystone - nova - cinder - neutron - glance - account control-nodes01 - keystone API - nova API - cinder API - neutron API - glance API control-nodes02 - keystone API - nova API - cinder API - neutron API - glance API OpenStack Management network step 1) step 2) step 3) step 4) public API: step 3), step 4) step 3) ext-api-wrapper0 [12], it is a php program. request URI and header, and the input value of json of the body was confirmed by php, and then call the real OpenStack API as the next processing. step 4) OpenStack API that is checked the input value will be run.
- 33. 35 OpenStack Juno cluster: •ConoHa (Juno) and Z.com cloud • AppsCloud (Juno)
- 34. 36 Tokyo Singapore Sanjose # ConoHahas data centers in 3 Locations
- 35. 37 Tokyo Singapole User/tenant User/tenant APIManagement Keystone API API Management Keystone APIAPI Management Keystone API Token Token Tokyo SanJoseSingapore API Management Keystone API API Management Keystone API READ/WRIT E READ READ TokenToken Token Do not create/delete users Do not create/delete users Our Customer base User administration # User-registration is possible in Japan only DB Replication DB Replication User/tenant User/tenantUser/tenant R/W R/W
- 36. 38 OpenStack Juno: 2service cluster, released Mikumo ConoHa Mikumo Anzu Mikumo = 美雲 = Beautiful cloud New Juno region released: 10/26/2015
- 37. 39 • Service model:Public cloud by KVM • Network: 10Gbps wired(10GBase SFP+) • Network model: – Flat-VLAN + Neutron ML2 ovs-VXLAN overlay + ML2 LinuxBridge(SaaS only) – IPv6/IPv4 dualstack • LBaaS: LVS-DSR(original) • Public API – Provided the public API (v2 Domain) • Compute node: ALL SSD for booting OS – Without Cinder boot • Glance: provided • Cinder: SSD NexentaStore zfs (SDS) • Swift (shared Juno cluster) • Cobbler deply on under-cloud – Ansible configuration • SaaS original service with keystone auth – Email, web, CPanel and WordPress OpenStack Juno: 2 service cluster, released • Service model: Public cloud by KVM • Network: 10Gbps wired(10GBase SFP+) • Network model: – L4-LB-Nat + Neutron ML2 LinuxBridge VLAN – IPv4 only • LBaaS: Brocade ADX L4-NAT-LB(original) • Public API – Provided the public API • Compute node: Flash cached or SSD • Glance: provided (NetApp offload) • Cinder: NetApp storage • Swift (shared Juno cluster) • Ironic on under-cloud – Compute server deploy with Ansible config • Ironic baremetal compute – Nexsus Cisco for Tagged VLAN module – ioMemory configuration
- 38. 40 OpenStack Cinder Blockstorage: ConoHa: NexentaStor(SDS) AppsCloud: NetApp
- 39. 41 NexentaStor zfs cinder:ConoHa cloud(Juno) Compute
- 40. 42 NetApp storage: GMOAppsCloud(Havana/Juno) If you are using the same Cluster onTAP NetApp a Glance and Cinder storage, it is possible to offload a copy of the inter-service of OpenStack as the processing of NetApp side. • Create volume from glance image ((glance the image is converted (ex: qcow2 to raw) required that does not cause the condition) • Volume QoS limit: Important function of multi- tenant storage • Uppper IOPS-limit by volume
- 41. 43 OpenStack Ironic: Only AppsCloud: •Undercloud Ironic deploy • Multi-tenant Ironic deploy
- 42. 44 Ironic with undercloud:GMO AppsCloud(Juno) For Compute server deployment. Kilo Ironic and All-in-one • Compute server: 10G boot • Clout-init: network • Compute setup: Ansible Under-cloud Ironic(Kilo): It will use a different network and Ironic Baremetal dhcp for Service baremetal compute Ironic(Kilo). (OOO seed server) Trunk allowed vlan, LACP
- 43. 45 Ironic(Kilo) baremetal: GMOAppsCloud(Juno) Boot baremetal instance • baremetal server (with Fusion ioMemory SanDisk) • 1G x4 bonding + Tagged allowed VLAN • Clout-init: network + lldp • Network: Nexsus Cisco Allowd VLAN security Ironic Kilo + Juno: Fine • Ironic Python driver • Whole Image write • Windows: OK
- 44. 46 • Service model:Public cloud by KVM • Network: 10Gbps wired(10GBase SFP+) • Network model: – Flat-VLAN + Neutron ML2 ovs-VXLAN overlay + ML2 LinuxBridge(SaaS only) – IPv6/IPv4 dualstack • LBaaS: LVS-DSR(original) • Public API – Provided the public API (v2 Domain) • Compute node: ALL SSD for booting OS – Without Cinder boot • Glance: provided • Cinder: SSD NexentaStore zfs (SDS) • Swift (shared Juno cluster) • Cobbler deply on under-cloud – Ansible configuration • SaaS original service with keystone auth – Email, web, CPanel and WordPress OpenStack Juno: 2 service cluster, released • Service model: Public cloud by KVM • Network: 10Gbps wired(10GBase SFP+) • Network model: – L4-LB-Nat + Neutron ML2 LinuxBridge VLAN – IPv4 only • LBaaS: Brocade ADX L4-NAT-LB(original) • Public API – Provided the public API • Compute node: Flash cached or SSD • Glance: provided (NetApp offload) • Cinder: NetApp storage • Swift (shared Juno cluster) • Ironic on under-cloud – Compute server deploy with Ansible config • Ironic baremetal compute – Nexsus Cisco for Tagged VLAN module – ioMemory configuration
- 45.
Editor's Notes
- #2 私からのお話は以上です。 ここからはGohkoさんが、我々のパブリッククラウドで使われている技術について解説します。 ============== Hi everyone. My name is Naoto Gokhko. I am working in the Cloud Service development team of GMO Internet. We are from a team at GMO Internet that focuses on developing services based on Openstack.
- #3 AND, we are offerring multiple public cloud services; ConoHa cloud, z.com cloud, GMO AppsCloud, Onamae cloud and vps.
- #4 Within all the services we’ve launched so far, we have 2,000 active physical node and over 100,000 VMs activated.
- #5 Development Team !!
- #6 I would like to talk about our current development team, and the scale of the development department. Neutron Team is 4 people. Cloud API Team is 5 people. Cloud Infra Team is 11 people. Application cloud Team is 5 people.
- #7 There is Neutron Team and Cloud API Team. And There is Cloud Infra Team and AppsCloud Team.
- #8 This photo was took made late at night. Here he who works has just joined the company after graduating from university last year. But, he is a great guy. He did a renovation of cinder driver of speed up.
- #9 We are limited number of people. But, we have to run a lot of OpenStack service clusters.
- #10 And here, we will talk about our service development history by OpenStack.
- #11 Our multiple OpenStack clusters are operating in multiple Products within our environment. Starting with the Diablo cluster. Then, share the OpenStack code that we have to fix bug, and we provieded to our group company in Vietnam. we’ve built many OpenStack clusters such as Grizzly for 1st VPS ConoHa, Havana for GMO AppsCloud, and Juno for ConoHa/z.com cloud/GMO AppsCloud, and they are still in operations. Swift cluster are shared every cluster.
- #12 It is a dark age for the cloud suppliers cloud cat is surprised. !! The cost to operate Multi version Openstack have increased, and it is difficult to upgrade or add new features. Managing multiple sites of OpenStack is a headache for us. ========= クラウド事業者の暗黒面に突入しています。
- #13 First, we describe the swift cluster.
- #14 The swift object nodes we use servers with twelve HDD of Asus company. I chose those of high-clock(3.3GHz) in about Xeon E3.
- #15 Network nodes, such as LVS-DSR and L7-proxy is using microblade of Supermicro company. In E3-Xeon, the CPU is the node of the clock-oriented.
- #16 The account-server and container server that I used things E5-2CPU model. Sqlite Database area is SSD. The swift-proxy nodes had to be output for charging the log of ceilometer agent. Log area is HDD in RAID 10.
- #17 This is a block diagram of a swift object storage. Load balancer is LVS-DSR and Layer7 HAProxy. The reason for using HAProxy is, we assume that the future put a pure HTTP request in caching node like a Apache Traffic Server.
- #18 I was upgrade to Juno in its own package the swift environment from havana. upgrade in python 2.6 is Juno is the last. In the next and later versions, I need to consider how to be what.
- #19 OpenStack Swift cluster is 5 zones 3 copy configuration.
- #20 As the first to say, swift cluster is shared by more than one service. As a result, the system has multiple API endpoint.
- #21 Multiple of Swift-Proxy settings are connected to each of the keystone. reseller_prefix has started with different settings as the name space. Other node of swift cluster has been completely shared.
- #22 I will explain what kind of transition of configuration was there in our service for the computing environment.
- #23 This is Diablo cluster. Still alive. We use Nova virtulization flat Nova network, keystone and glance.
- #24 We challenged the ovs-GRE overlay networking AND IPv6/IPv4 dualstack by Quantam in Grizzly release. But we need to many many many fix, a lot of the program. As a result, We were not able to publish the API for End user.
- #25 First(1st) ovs-GRE overlay network was giant in L2 networking. Grizzly’s overlay configuration is Full Mesh. But, sometime the trouble has occurred in this GRE-mesh-tunnel.
- #26 Network traffic of the Game is the load is large. The current mobile game requires both a large amount of data transfer and short packet forwarding is. It is using Brocade, Inc. appliance as LBaaS. We're using the HPE 3 PAR and NetApp FAS as Cinder. We have provided as the first public API in this service the OpenStack API to the end user.
- #27 Neutron LinuxBridge model is very Fast, This is simple is the Best. This cloud is optimized services of the GAME servers.
- #28 Allowed VLAN to trunked port is allowed only VLAN to be used in LinuxBridge in Compute node. By controlling the dynamic that allowed vlan, it will be the performance of the network switch can be used in a more L2 network. So Neutron LinuxBridge VLAN, VM and nova baremetal instance is almost the same as to control the trunk allowed vlan of Switch Port, to build the security of the tenant network.
- #29 By using this python library for the above of Cisco iOS, we have to control the NX-OS.Processing is serialized as API runs on the Switch.
- #30 The nova-baremetal environment of havana, has major processing are modified to ansible is executed. Even in our Juno of Ironic environment, as well as to configure the network with link aggregation in ansible.
- #31 This is ALL component and networks. Differences in Juno environment, and that the version of OpenStack is different, it is that the nova-baremetal has changed to Ironic.
- #32 I will explain about the security and load balance of the public API.
- #33 For public API are running on the same system configuration from the time of Havana. LVS-DSR (L4), reverse-proxy (L7), API wrapper program (L8), has become a three-layer configuration that.
- #34 step 1) LVS-DSR (L4) is received https(tcp/443) packet, then forward api-reverse-proxy real IP’s. step 2) HAProxy has valid API ACL and backend server configurations. IF HAProxy allowed POST “/v2.0/tokens”, then the request call to ext-api-wrapper0[12]. Layer-7 reverse proxy, a confirmation of permission URI with ACL, method verification, and set the request limit setting. As a result, it is possible to protect from DoS and DDoS of the API, you can be a back-end API server in a secure configuration.
- #35 step 3) ext-api-wrapper0 [12], it is a php program. request URI and header, and the input value of json of the body was confirmed by php, and then call the real OpenStack API as the next processing. step 4) OpenStack API that is checked the input value will be run.
- #36 The last is a cluster of Juno. We will talk about ConoHa and AppsCloud.
- #37 ConoHa has 3 reageon cluster. First, ConoHa has 3 of the region. Supporting multi region was our first priority out of all the features in the roadmap. Physical location of the servers means a lot to our users. The data center location we chose initially were Tokyo, Singapore and San Jose. We’ve successfully built a multi region architected OpenStack environment between the 3 locations.
- #38 The system that manages both service site and user information only exists in Japan. Therefore user registration was only available in Japan.
- #39 Among them, it has built a AppsCloud in Juno Tested. The AppsCloud of Juno environment was able to build in the minimum of time.
- #40 Left side is ConoHa, Right side is AppsCloud. ConoHa is cheap price cloud. AppsCloud is high performance cloud.
- #41 For Cinder, we mentioned the difference in ConoHa and AppsCloud.
- #42 NexentaStor is a simple an SDS product due to the zfs, it is OpenIndiana system. The collaboration between Dell-Nexenta, you can build the structure at a low price was taking advantage of the performance of the SSD.
- #43 Meaning that use NetApp as Cinder is because it is possible to provide a high speed due to copy offload function to volume expansion from the glance image.
- #44 About Ironic in Juno environment, and easy to add the information.
- #45 In Juno environment, we are using a Ironic and Ansible to the initial construction of the node. All in one OpenStack AND set up the OS with a GUI in Ironic, at the final networking configuration: bonding, VLAN and OpenStack software environment in ansible.
- #46 In AppsCloud, it offers baremetal node in Ironic as Guest environment. Security settings of tenant network will be applied by the previous switch / port driver. ansible By execution in Ironic, setting of PCI-E Flash, such as ioMemory is also possible.
- #47 In this way, in a different offer price range at Juno, we were able to be provided as a public service in a different cost. In particular, ConoHa as OpenStack service, we will continue to improve the compatibility of the provision of the OpenStack API for developers. ========================== このように、Junoにて異なる提供価格帯で、違うコストでpublicサービスとして提供することができました。 とくに、OpenStackサービスとしてのConoHaは、開発者のためのOpenStack APIの提供の互換性について継続的に改善していきます。
- #48 That’s all from me. Thanks.