Building complete private cloud architectures that support both traditional application virtualization and emerging distributed cloud systems requires compute and object storage storages. Together Apache CloudStack and Basho's Riak CS provide the infrastructure to support both of workloads. The presentation covers the following topics: 1. Why Private Cloud? 2. Anatomy of a Private Cloud 3. Building a Apache CloudStack Compute Offering 4. Large Object Storage using Riak CS 5. Your Own Private Cloud: The Riak CS Apache CloudStack Integration Roadmap This presentation was originally delivered to the Silicon Valley Riak group on 27 June 2013.

1. MAKING CLOUDY PEANUT
BUTTER CUPS: APACHE
CLOUDSTACK + RIAK CS
John Burwell
(jburwell@basho.com | @john_burwell)
Thursday, June 27, 13

2. Who Am I
• Consulting Engineer @ Basho Technologies
• Apache CloudStack PMC Member
• Ran operations and designed automated provisioning for hybrid
analytic/virtualization clouds
• Led architectural design and server-side development of a SaaS
physical security platform
Thursday, June 27, 13

3. Thursday, June 27, 13

4. Hopefully not ...
Thursday, June 27, 13

5. Thursday, June 27, 13

6. Thursday, June 27, 13

7. Thursday, June 27, 13

8. Composable Infrastructure
• Resource Pooling
• Rapid Elasticity
• Self Service
• Metering/Charge Back
Thursday, June 27, 13

9. Doesn’t vSphere/
XCP/XenServer
already do that?
Thursday, June 27, 13

10. Traditional Workload
Management
Network
Hypervisor
Cluster
Hypervisor
Cluster
Shared

11. Storage
• Reliable
• Fault tolerant
• Specialized hardware
• IT managed provisioning
• Static scale out model
Increased TCO; Decreased organizational agility
Thursday, June 27, 13

12. Cloud Workload
• Reliable
• Fault Tolerant
• Commodity Hardware
• Self-Service provisioning
• Dynamic scale out model
SDN
Rack Rack
Object

13. Storage
App App
Decreased TCO; Increased
organizational agility
Thursday, June 27, 13

14. Traditional
Workload
Cloud
Workload
The Chicken Or The Egg?
Thursday, June 27, 13

15. Got It ... Why Go Private?
• Leverage existing infrastructure investment
• Customization and Control
• Regulatory/Security Requirements
• Cost
Thursday, June 27, 13

16. Open Source Private Cloud
+
Compute Services Object Storage Services
Thursday, June 27, 13

17. Make infrastructure boring.
Thursday, June 27, 13

18. What Is Apache CloudStack?
§Open source software that deploys
and manages large networks of
virtual machines, as a highly
available, highly scalable
Infrastructure as a Service (IaaS)
cloud computing platform.
§CloudStack is a Top-Level Project at
the Apache Software Foundation.
Thursday, June 27, 13

19. Apache CloudStack Users
Thursday, June 27, 13

20. Infrastructure Model
• Management Server Farm
• Management and provisioning tasks
• Zone
• Collection of pods, network offerings and secondary storage
• Pod
• Collection of clusters in the same failure boundary
• Cluster
• A grouping of hosts and their associated storage
• Hosts
• Servers onto which services will be provisioned
• Primary Storage
• VM disk storage
• Network
• Logical network associated with service offerings
• Secondary Storage
• Template, snapshot and ISO storage
Zone
CloudStack Pod
Cluster
Host
Host
Network
Primary
Storage
VM
VM
CloudStack Pod
Cluster
Secondary
Storage
Thursday, June 27, 13

21. Management Server Farm
• MS is stateless. MS can be
deployed as physical server
or VM
• Single MS node can manage
up to 10K hosts. Multiple
nodes can be deployed for
scale or redundancy
• RHEL/CentOS (5.4+), Ubuntu
(10.0.4+), Fedora (16+)
User

22. APIAdmin

23. API
Load
Balancer
Management
Server
Management
Server
MySQL
MySQL
Replica
Thursday, June 27, 13

24. Got Scale?
• Running in production at 30,000 physical hosts, supported by only 4
management server instances
• Even greater scale by using CloudStack in a Regional model
• Remember: The separation of control, management and data planes is
critical for cloud platform scale.
Thursday, June 27, 13

25. Deployment Architecture
• Hypervisor is the basic unit of scale.
• Cluster consists of one ore more hosts of
same hypervisor
• All hosts in cluster have access to shared
(primary) storage
• Pod is one or more clusters, usually with
L2 switches.
• Availability Zone has one or more pods,
has access to secondary storage.
• One or more zones represent a cloud
Pod
1
….
Cluster
N
L2
Host 2
Cluster
1
Host 1
Primary
Storage
Zone
1
….
L3
Secondary
Storage
Pod
N
Management
Server Farm
Internet
Thursday, June 27, 13

26. Hypervisors
• Xen / XCP (Ubuntu 12.04.2)
• XenServer (6.1)
• KVM (Ubuntu 12.04.2 / CentOS 6)
• VMWare vCenter (4.1 / 5.1)
Thursday, June 27, 13

27. Storage
• ISCSI
• NFS
• Local Filesystem Storage
• Ceph
• S3-backed Secondary Storage
• Swift-backed Secondary Storage
Thursday, June 27, 13

28. Networking
• VLAN and Security Group isolation
• Load Balancer Support
• Juniper SRX, F5, Nicira NVP and Open vSwitch integration
Thursday, June 27, 13

29. 4.1: Hot Off The Presses
• AWS style regions
• Event pub-sub framework
(RabbitMQ implementation)
• Advanced search within the UI
• API Server request throttling
• API Discoverer Service
• Self-Service Password Reset
• Users directly changing their
API keys
• EC2 query API
• Cloudmonkey CLI
• Autoscale (F5-specific)
Much, much more ....
Thursday, June 27, 13

30. 4.2: On-Deck
• Midonet and BigSwitch SDN integrations
• Enhanced Baremetal provisioning
• VM I/O Throttling
• Hyper-V 2012 Support
• LXC Support
• Cisco VSG integration
• Cisco ASA 1000V
• VM Affinity Rules
• VM Affinity Rules
• Zone-wide primary storage
• Security group isolation in Advanced Network
zones
• Dedicating resources to domains and accounts
• IP Address reservation w/o a vNIC allocation
• vSphere DVS support
• UI Plugin framework
Thursday, June 27, 13

31. First A Little Riak
• Dynamo-inspired, masterless key-value store
• Apache License
• Always available for writes
• Highly available reads
• Secondary Indexes/Search
• MapReduce
• Operations Friendly
Thursday, June 27, 13

32. ... Then A Little Riak CS
• Built on top Riak
• Apache Licensed
• S3-compatible API
• Large object support and multipart upload
• Multi-tenancy and per-user reporting
• Operations Friendly
Thursday, June 27, 13

33. How It Works
Large

34. Object
Riak

35. CS Riak

36. CS Riak

37. CS
1

38. MB 1

39. MB 1

40. MB 1

41. MB 1

42. MB 1

43. MB 1

44. MB 1

45. MB
Riak Riak Riak
1

46. MB
Thursday, June 27, 13

47. Riak CS Roadmap
• Keystone integration
• OpenStack Storage API v1 (aka Swift) support
• Performance improvements around large bucket operations
Thursday, June 27, 13

48. The Complete Private Cloud
Management
Network
Hypervisor
Cluster
Hypervisor
Cluster
Shared

49. Storage
SDN
Rack Rack
Riak

50. CS
App App
CloudStack

51. Zone CloudStack

52. Zone
Thursday, June 27, 13

53. Apache CloudStack + Riak CS Roadmap
• Now
• S3-backed Secondary Storage
• 4.2
• Partial support for using object stores as secondary storage
• 4.3
• Integrated authentication
• Complete support for using object stores as secondary storage
Thursday, June 27, 13

54. Join Our Community
• Learn more @ http://cloudstack.apache.org
• Grab it @ http://cloudstack.apache.org/downloads.html
• Follow us @cloudstack
• Help from the cloudstack-users@cloudstack.apache.org mailing
list or on Freenode @ #cloudstack
• Contribute on cloudstack-dev@cloudstack.apache.org mailing
list or on Freenode @ #cloudstack-dev
Thursday, June 27, 13

55. Up And Running With Riak CS
• Learn more @ http://basho.com/riak-cloud-storage
• Grab a distribution @ http://docs.basho.com/riakcs/latest/
riakcs-downloads/
• Follow us @basho
• Help from the riak-users@lists.basho.com mailing list or on
FreeNode @ #riak
• Contribute @ http://github.com/basho/riak_cs
Thursday, June 27, 13

56. Thoughts? Questions?
Thursday, June 27, 13

57. Thank you!
Slides available @ http://speakerdeck.com/jburwell
Thursday, June 27, 13