Some Performance Tuning Know-How about CloudStack and the future plan about more scalable CloudStack platform.

1. IDC Frontier Inc. All rights reserved.
Toward Scalable and powerful CloudStack
IDC Frontier, Inc.
Takashi Kanai
Sep, 26th, 2018

2. 2
IDC Frontier Inc. All rights reserved.
Agenda
• Introduction of myself & company
• Performance of list** API
• Performance of updating VR policy
• Scalability of CloudStack Zone with VMware

3. 3
IDC Frontier Inc. All rights reserved.
About me
Name : Takashi Kanai
Company : IDC Frontier, Inc.
IaaS Group Leader, SRE Dept, Cloud Unit.
- Infrastructure Designing, Deploying, etc… of IaaS Service from
2008.
- Provide IaaS service using CloudStack 2.x from 2011
- Renewed IaaS service using CloudPlatform 4.x from 2014
MVP : Cloud and Data Center Management (2017/03-)
SNS :
• Facebook https://www.facebook.com/anikundesu
• Blog http://www.takanyan.net/
• Twitter @anikundesu
• LinkedIn https://jp.linkedin.com/in/takashikanai/ja
• SlideShare ttps://www.slideshare.net/anikundesu/presentations

4. 4
IDC Frontier Inc. All rights reserved.
Overview of IDC Frontier (IDCF)
IDC Frontier is,
Datacenter provider and
Cloud infrastructure provider.
Becomes subsidiary of
SoftBank KK from this May
【Data Center】
Metropolitan, NorthEast, and East Japan DC.
Top level share in Japanese DC market.
【Cloud (IaaS based on CloudStack】
Japan East region/Japan West Region

5. 5
IDC Frontier Inc. All rights reserved.
History of IDCF

6. 6
IDC Frontier Inc. All rights reserved.
IDCF Operated Data Centers
Kukai
(GREEN 500 No.2
@2017,summer)

7. 7
IDC Frontier Inc. All rights reserved.
IDCF Cloud : Simple Powerful
Server created in only 20 seconds.
There is no limit to network bandwidth.
Access with large capacity 10G
connection.
Minimum 1 CPU, 1 GB memory
specifications from 500 yen to
Dedicated type/GPU instance.
Excellent cost performance.
Easy cloud management with
simple and easy to use UI and API.
Online support and invoicing are
also available, making
it ideal for business use.

8. 8
IDC Frontier Inc. All rights reserved.
Progress of IDCF Cloud

9. 9
IDC Frontier Inc. All rights reserved.
Performance of list** API

10. 10
IDC Frontier Inc. All rights reserved.
Struggling with Slow Response of listTemplates
Slow response when templatefilter is featured, community, or executable.
Definition of Template_view was not correct and return many fault and duplicated records.
We fix the definition by re-creating template_view
Fix
template_view
definition
Version Up
to 4.5.1
Add Templates

11. 11
IDC Frontier Inc. All rights reserved.
Tunig Global Config to improve DB performance
Database Query # of queries in 10 min Tuning point
SET autocommit= 2295960
SELECT 1 1074006
SELECT @@session.tx_isolation 1073917
SELECT vm_instance.id, vm_instance.name, vm_instance.vnc_password 341837Ping Interval 180 ⇒ 240
SELECT user_vm_details.id, user_vm_details.vm_id, user_vm_details.name 221194capacity.check.period 300 ⇒ 3600
commit 149414
rollback 146534
SELECT networks.id, networks.mode, networks.broadcast_domain_type 119965router.check.interval 600 ⇒ 900
SELECT host.id, host.disconnected, host.name, host.status, host.type 74702direct.agent.scan.interval,host.stats.interval
SELECT nic_secondary_ips.id, nic_secondary_ips.nicId, nic_secondary_ips.domain_id, 72913
SELECT ntwk_service_map.id, ntwk_service_map.network_id, 55226
SELECT configuration.instance, configuration.component, configuration.name, 42369
SELECT resource_tag_view.id, resource_tag_view.uuid, resource_tag_view.key 33057
SELECT op_host_planner_reservation.id, op_host_planner_reservation.host_id, 19536
SELECT account.id, account.account_name, account.type, account.domain_id, 10639
SELECT cluster_details.id, cluster_details.cluster_id 10004
Investigating whole queries to CloudStack DB

12. 12
IDC Frontier Inc. All rights reserved.
Result of tuning global config
1,800 “select” queries/sec reduce (about 30%)
Average Response time 4.15s ⇒ 0.90 s
1,800

13. 13
IDC Frontier Inc. All rights reserved.
Performance of
updating VR policies

14. 14
IDC Frontier Inc. All rights reserved.
FW rule and Port Forwarding rule add to IP
・Adding FW rules to same IP is fast but become longer
・Adding Port Forwarding rules to same IP becomes very slower
Biggest customer use(~30s) Biggest customer use(~1min)
M2 = 2vCPU, 2GB Memory. (version 4.7.0)

15. 15
IDC Frontier Inc. All rights reserved.
Adding vCPUs to VR
・Adding FW rules become faster when vCPU added
・Adding Port Forwarding rules doesn’t become faster
We tell customers to add more IPs
and divide Port Forwarding rules.
XL8 = 8vCPU, 8GB Memory. (version 4.7.0)

16. 16
IDC Frontier Inc. All rights reserved.
Scalability of CloudStack Zone
with VMware

17. 17
IDC Frontier Inc. All rights reserved.
Zones of IDCF Cloud(2018/05)
West Region East Region 1
East Region 2
zone
zone
zone

18. 18
IDC Frontier Inc. All rights reserved.
Scalability bottleneck
Limitation
vCenter / Zone 1
Hosts / Zone
Limited by vShere
Configuration Maximums
VMs / Zone
Datastores / Zone
etc…. / Zone
Number of vCenter Server per Zone is critical bottleneck for Scalability

19. 19
IDC Frontier Inc. All rights reserved.
Idea for multiple vCenter support
･Multiple vCenter in one Zone
･Zone-wide primary storage is migrated to “vCenter-wide” primary storage

20. 20
IDC Frontier Inc. All rights reserved.