- PCextreme is a Dutch hosting company that runs a public cloud on Apache CloudStack with Ceph storage, serving over 50,000 customers. - Their public cloud infrastructure consists of over 2,000 running instances across multiple zones and regions, powered by 44 hypervisors with 11TB of RAM and 704 cores. - They use Ceph as the primary storage, which provides 700TB of scalable and fault tolerant storage across 52 servers, and have integrated it seamlessly into their CloudStack deployment.

1. A public cloud on Ceph and
Apache CloudStack

2. Who am I?
● Wido den Hollander (1986)
● Co-owner and CTO @ PCextreme B.V.
● Ceph community member
● CloudStack committer and PMC member
● Wrote the Ceph (RBD) integration
● Work on the KVM Hypervisor code
● Maintain Ubuntu .deb packages

3. Who is PCextreme B.V.?
● Dutch hosting company since 2004
● 25 employees
● 50.000 customers
● >100.000 shared hosting customers
● Running a public cloud on Apache CloudStack
with Ceph storage
● Servers in Amsterdam, Rotterdam, Miami, Tokyo
and Los Angeles

4. This is not a commercial talk
I am just showing how great CloudStack works
for my company
I have €20 gift cards to give away

5. Our public cloud: Software
● Apache CloudStack 4.4.1
● KVM hypervisors on Ubuntu 14.04
● Ceph Primary Storage
● NFS Secondary Storage

6. Our public cloud: Hardware
● 100% SuperMicro hardware
● Intel Xeon E5 processors with 256GB of RAM
per hypervisor
● Tiered SSD and HDD storage with Ceph
● 10Gbit networking

7. Our public cloud: Hypervisors

8. Our public cloud: The numbers
● 2.000 Running Instances
● 2 zones in Amsterdam
● Miami, Tokyo and Los Angeles within a month
● 44 Hypervisors
● 11TB of RAM
● 704 cores
● 700TB of Ceph storage
● 30.000 API calls per day
● Customers have API access

9. Our public cloud: Network
● Basic networking
● Keep it simple
● Let real routers do the routing
● Public Clouds get a lot of DDoS attacks...
● Security Grouping for isolation
● IPv4 and IPv6
● No private networks, all public IP-space
● IPv6 is currently not fully supported by
CloudStack

10. Our public cloud: Ceph
● Ceph is a Open Source unified distributed
storage platform which can expose Block
Devices
● Integrated into various clouds
● We use it as Primary Storage for our cloud

11. Our public cloud: Ceph
● Running 52 Ceph servers spread out over three
racks
● Roughly 700TB of RAW storage
● Mix between 1TB SSD and 3TB HDD
● 10Gbit network for low-latency

12. Our public cloud: Ceph
● Running 52 Ceph servers spread out over three
racks
● Roughly 700TB of RAW storage
● Mix between 1TB SSD and 3TB HDD

13. Our public cloud: Ceph

14. Our public cloud: Ceph

15. Our public cloud: Ceph
● Ceph is has no Single Point of Failure. If fault
tolerant and self-healing.
● We never schedule a maintenance window and
do maintenance during office hours.
● Ceph can scale from a couple of TB into multiple
PBs without downtime
● We started with 7 machines and grew to 52
without downtime

16. Our public cloud: Scaling
● The main problem with scaling is storage
● Ceph solves this problem for us
● Adding Hypervisors is very simple
● Instantly provides more RAM and CPU
● Live Migrate Instances to new hardware

17. Why CloudStack?
It works

18. Why CloudStack?
● Open project, driven by the Apache Foundation
● Easy access to the project and code
● Stable code and understandable code
● Very rich set of features
● Fast and rich API
● And it just works

19. Did we encounter issues?
● Yes, we did. None of them were very serious:
● High Availability kicked in wrongly a couple of
times causing some Instances to be restarted
● Firewalling issue which caused some Instances
to loose their network
● We fixed them upstream

20. Stamina vs Agile
● Stamina Instances are Highly Available and use
Ceph storage
● For persistent storage and long running
applications
● Agile Instances are running on local SSD
storage and are not HA
● For short-running applications, eg webservers

21. Stamina vs Agile

22. Stamina vs Agile
● Using different Service Offerings and Host tags
we deploy the Instances on the right hardware

23. Our Control Panel
● We wrote our own Control Panel which talks to
the CloudStack API
● Custom made for our customers
● Developed in-house
● Written in PHP (Laravel framework)
● Scales on mobile devices
● URL: https://cp.pcextreme.nl/

24. Our Control Panel

25. Our Control Panel

26. Conclusion
● CloudStack has been working for over 2 years
for us
● Reliable and easy to maintain

27. Thank you!
● I have gift cards to try out our public cloud
● Twitter: @widodh
● E-Mail: wido@widodh.nl
● Blog: http://blog.widodh.nl/
● https://www.pcextreme.com/