The document presents an overview of Apache CloudStack 4.11, detailing new features, improvements, and operational capabilities resulting from months of community effort. Key highlights include support for multiple disk OVA files, enhanced LDAP mapping for multi-tenancy, and improved user and operator features. The update emphasizes automation, enhanced integrations, and internal upgrades aimed at better cloud infrastructure management.

1. The Cloud Specialists
Apache CloudStack 4.11
ShapeBlue.com • @ShapeBlue
Paul Angus, VP Technology • @CloudyAngus
paul.angus@shapeblue.com
B e h i n d T h e H e a d l i n e s

2. The Cloud Specialists
ShapeBlue.com @ShapeBlue
“ShapeBlue are expert builders of public
& private clouds. They are the leading
independent global CloudStack services
company”
A b o u t S h a p e B l u e

3. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
Paul Angus – VP Technology
• Global authority on CloudStack & cloud infrastructure design.
• 20+ years C-Level experience.
• Apache CloudStack project committer & PMC member
• Specialising in deployment of CloudStack
and surrounding infrastructure especially the user story
• Autodesk, SAP, British Telecom, Georgian Ministry of Justice,
Orange, TomTom, PaddyPower, Ascenty, BSkyB
A b o u t M e

4. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
S h a p e B l u e C u s t o m e r s

5. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
S h a p e B l u e C u s t o m e r s

6. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
S h a p e B l u e C u s t o m e r s

7. The Cloud Specialists
ShapeBlue.com @ShapeBlue
IntroductionIntroduction

8. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Overview
• User Features
• Operator Features
• Integrations
• CloudStack Internals
A g e n d a

9. The Cloud Specialists
ShapeBlue.com @ShapeBlue
OverviewOverview

10. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Released 29th January
• Culmination of 8 months of community work from 30 different
sources
• 100s of updates
• 30+ New Features
• Best automated test coverage yet
O v e r v i e w

11. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
O v e r v i e w
CLOUDSTACK-9813 Use configdrive for userdata, metadata & password
CLOUDSTACK-4757 Support OVA files with multiple disks for templates
CLOUDSTACK-10109 Enable dedication of public IPs to SSVM and CPVM
CLOUDSTACK-10109 Enable dedication of public IPs to SSVM and CPVM
CLOUDSTACK-10117 LDAP mapping on domain level

12. The Cloud Specialists
ShapeBlue.com @ShapeBlue
Questions?Questions?

13. The Cloud Specialists
ShapeBlue.com @ShapeBlue
User FeaturesUser Features

14. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• New Network Type (L2)
• Multi-tenant LDAP (LDAP mapping on domain level)
• Bypass secondary storage template copy/transfer for KVM
• Several UI Improvements.
• Support OVA files with multiple disks for template
• Use configdrive for userdata, metadata & password
U s e r F e a t u r e s

15. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Support for Secondary IPv6 Addresses and Subnets.
• IPv6 Prefix Delegation support in basic networking.
• Allow more than 7 NICs to be added to a VMware VM.
• Usage metrics for VM snapshot on primary storage.
U s e r F e a t u r e s - U I

16. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• No Virtual Router
• No IP addresses
• Simple Isolated network
L a ye r 2 N e t wo r k s
L2 Network
User Instance
VM
User Instance
VM: AD + DHCP + DNS
User Instance
VM

17. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Enables ‘User-Managed’
networks
L a ye r 2 N e t wo r k s
L2 Network
User Instance
VM
User Instance
VM: AD + DHCP + DNS
User Instance
VM
User Instance
VM (ie jumpbox)
VR

18. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Enables Service-Chaining
L a ye r 2 N e t wo r k s
CloudStack VR
L2 Network
User Instance
vRouter/Firewall/VPN
L2 Network
User Instance
VM
User Instance
VM: AD + DHCP + DNS
L2 Network
User Instance
Web server
DMZ

19. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
L D A P M u l t i - Te n a n c y

20. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Support for metalinks including
preferred servers
• Direct Download templates not stored
on Secondary Storage, only a pointer
• If host cannot reach template location,
download is attempted on alternate
host
B y p a s s S e c o n d a r y S t o r a g e

21. The Cloud Specialists
ShapeBlue.com @ShapeBlue
Operator FeaturesOperator Features

22. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Dedication of public IPs to SSVM and CPVM
• Support for separate subnet for SSVM and CPVM
• Annotation feature for CloudStack entities such as hosts
• Improve available information in UI for admins
• Limit admin access from specified CIDRs
• Expansion of Management IP Range
O p e r a t o r F e a t u r e s

23. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Move user across accounts.
• Ability to specify mac address when deployVirtualMachine or
addNicToVirtualMachine is called
• Expire VM snapshots after configured duration
• Usage metrics for VM snapshot on primary storage.
• Support for XenServer 7.1 and 7.2, and improved support for
VMware 6.5
O p e r a t o r F e a t u r e s

24. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
S e p a r a t e S u b n e t & V L A N f o r C P V M a n d S S V M
• Warum? – PCI Compliance.
• If not separated, then guest
networks become in-scope and
must be audited.

25. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• CPVM & SSVM can be on a
separate VLAN/Subnet to
VRs on Public and
Management interfaces
S e p a r a t e S u b n e t & V L A N f o r C P V M a n d S S V M

26. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
l i s t A n n o t a t i o n s ( H o s t I m p l e m e n t a t i o n )
listHosts
Response Name Description Response Name
id the ID of the host id
annotation the last annotation set on this host by an admin annotation

27. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
listAnnotations (Admin only)
A n n o t a t i o n s ( F r a m e wo r k )
Parameter Name Description Required
entityid the id of the entity for which to show annotations False
entitytype the entity type False
id the id of the annotation False

28. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
U I I m p r o v e m e n t s

29. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
U I I m p r o v e m e n t s

30. The Cloud Specialists
ShapeBlue.com @ShapeBlue
IntegrationsIntegrations

31. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• CloudStack metrics exporter for Prometheus.
• Cloudian Hyperstore connector for CloudStack.
• Extra DHCP options support (Nuage Networks).
• Nuage VSP 5.0 support and caching of NuageVsp ID’s.
• Nuage domain template selection per VPC and support for network
migration.
• Enable NetScaler inline mode.
I n t e g r a t i o n s

32. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• NCC (NetScaler Control Center) integration in CloudStack
• Managed storage enhancements.
I n t e g r a t i o n s

33. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
List of metrics exported per zone:
- Total hosts
- Online hosts
- Offline hosts
- Per host:
- CPU speed Used
- CPU speed Total
- RAM Used
- RAM Total
- Total VMs running on host
- CPU cores Used
- CPU cores Total
- CPU speed Allocated for zone
- CPU cores Allocated for zone
- RAM Allocated for zone
- VMs (count in all states)
- Volumes Ready
- Volumes Destroyed
- Volumes Total
- Storage Pools
(Primary/Secondary)
- Disk size allocated (only for
primary storage)
- Disk size total
- Disk size used
- Private IP allocated
- Private IP total
- Public IP addresses allocated
- Public IP addresses total
- Shared Network IPs total
- Shared Network IPs allocated
- VLAN Allocated
- VLAN Total
- CloudStack cpu cores limit
(summation across domains)
- CloudStack memory limit
(summation across domains)
P r o m e t h e u s

34. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
SAMPLE
# Cloudstack Prometheus Metrics
cloudstack_hosts_total{zone="Sandbox-simulator",filter="online"} 4
cloudstack_hosts_total{zone="Sandbox-simulator",filter="offline"} 0
cloudstack_hosts_total{zone="Sandbox-simulator",filter="total"} 4
cloudstack_vms_total{zone="Sandbox-simulator",filter="starting"} 0
cloudstack_vms_total{zone="Sandbox-simulator",filter="running"} 2
cloudstack_vms_total{zone="Sandbox-simulator",filter="stopping"} 0
cloudstack_vms_total{zone="Sandbox-simulator",filter="stopped"} 0
cloudstack_vms_total{zone="Sandbox-simulator",filter="destroyed"} 0
cloudstack_vms_total{zone="Sandbox-simulator",filter="expunging"} 0
cloudstack_vms_total{zone="Sandbox-simulator",filter="migrating"} 0
cloudstack_vms_total{zone="Sandbox-simulator",filter="error"} 0
cloudstack_vms_total{zone="Sandbox-simulator",filter="unknown"} 0
cloudstack_vms_total{zone="Sandbox-simulator",filter="shutdowned"} 0
cloudstack_volumes_total{zone="Sandbox-simulator",filter="ready"} 2
cloudstack_volumes_total{zone="Sandbox-simulator",filter="destroy"} 0
cloudstack_volumes_total{zone="Sandbox-simulator",filter="total"} 2
cloudstack_storage_pool_gibs_total{zone="Sandbox-
simulator",name="PS0",address="10.147.28.6:/export/home/sandbox/primary0",type="
primary",overprovisioningfactor="2.0",filter="used"} 0.00
cloudstack_storage_pool_gibs_total{zone="Sandbox-
simulator",name="PS0",address="10.147.28.6:/export/home/sandbox/primary0",type="
primary",overprovisioningfactor="2.0",filter="allocated"} 0.00
export/home/sandbox/secondary",type="secondary",filter="total"} 0.00
cloudstack_private_ips_total{zone="Sandbox-simulator",filter="allocated"} 6
cloudstack_private_ips_total{zone="Sandbox-simulator",filter="total"} 199
cloudstack_public_ips_total{zone="Sandbox-simulator",filter="allocated"} 2
cloudstack_public_ips_total{zone="Sandbox-simulator",filter="total"} 199
cloudstack_shared_network_ips_total{zone="Sandbox-simulator",filter="allocated"} 0
cloudstack_shared_network_ips_total{zone="Sandbox-simulator",filter="total"} 0
cloudstack_vlans_total{zone="Sandbox-simulator",filter="allocated"} 0
cloudstack_vlans_total{zone="Sandbox-simulator",filter="total"} 101
cloudstack_domain_limit_cpu_cores_total 0
cloudstack_domain_limit_memory_mibs_total 0
P r o m e t h e u s

35. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Provide ease in connector configuration using
CloudStack global settings
• Perform SSO from CloudStack UI into Cloudian
Management Console (CMC) when the connector is
enabled
• Auto provisioning and de-provisioning of CloudStack accounts and domains
as Cloudian users and groups respectively
• During CloudStack UI logout, logout user from CMC
• CloudStack account will be mapped to Cloudian Users, and CloudStack
domain will be mapped to Cloudian Groups
C l o u d i a n H yp e r s t o r e I n t e r g r a t i o n

36. The Cloud Specialists
ShapeBlue.com @ShapeBlue
CloudStack InternalsCloudStack Internals

37. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Host-HA framework and HA-provider for KVM hosts with and NFS as
primary storage, and a new background polling task manager.
• Secure agents communication: new certificate authority framework
and a default built-in root CA provider.
• Separation of volume snapshot creation on primary storage and
backing operation on secondary storage.
• Embedded Jetty and improved CloudStack management server
configuration.
C l o u d S t a c k I n t e r n a l s

38. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Improved support for Java 8 for building artefacts/modules,
packaging, and in the systemvm template.
• A faster console proxy start-up and service availability.
• A new Debian 9 based systemvm template that patches systemvm
without requiring reboot.
C l o u d S t a c k I n t e r n a l s

39. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
Without Host HA
• When the connection to a CloudStack KVM host agent is lost,
this could be due to:
• Host agent crash
• Management network issue
• Host crash
• Which situation we are in cannot be determined
• Guest VMs may still be running. Split brain is VERY BAD™
- so do nothing…
H o s t H A

40. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
With Host HA
• Use other hosts in the cluster to poll shared storage.
• If no disk activity seen from VMs on the suspect host, then
reboot host (via IPMI)
• If still no connection from agent and no disk activity from VMs,
then power off host (via IPMI) and trigger VM HA
• All timeouts and checks are configurable.
H o s t H A

41. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
H o s t H A

42. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
• Framework to store and use SSL
certificates with CloudStack’s reach.
• CloudStack can act as a CA and
create/revoke certificates
• Implemented in host <-> mgmt. server
communications and upcoming TLS KVM
migration work
C A ( C e r t i f i c a t e A u t h o r i t y) F r a m e wo r k

43. The Cloud Specialists
ShapeBlue.com @ShapeBlue
SummarySummary

44. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
O v e r v i e w
CLOUDSTACK-9813 Use configdrive for userdata, metadata & password
CLOUDSTACK-4757 Support OVA files with multiple disks for templates
CLOUDSTACK-10109 Enable dedication of public IPs to SSVM and CPVM
CLOUDSTACK-10109 Enable dedication of public IPs to SSVM and CPVM
CLOUDSTACK-10117 LDAP mapping on domain level

45. The Cloud Specialists
ShapeBlue.com @ShapeBlue
Questions?Questions?

46. The Cloud Specialists
Apache CloudStack 4.11
https://www.slideshare.net/ShapeBlue/whats-new-in-cloudstack-411-behind-the-headlines
ShapeBlue.com • @ShapeBlue
Paul Angus, VP Technology • @CloudyAngus
paul.angus@shapeblue.com
B e h i n d T h e H e a d l i n e s