This document summarizes key considerations and best practices for planning a private cloud, based on lessons from companies like Zynga, telcos, and others. It discusses defining service offerings and tenant requirements, choosing the right virtualization infrastructure and storage options, optimizing operations, and more. The goal is to build a private cloud that provides agility, flexibility, and cost efficiency compared to public cloud options.

1. Planning your private cloud
Learning from the lessons of others
CloudStack Collaboration Conference Europe 2013
Tim Mackey – XenServer Community Evangelist

2. Private Cloud, Why Now?
• Valid alternative to public clouds that are cheap
and readily available
• Speed and agility of deployment
• Control of corporate assets
• Cloud Management Platform market maturity
• Future-proofing for nextgen, webscale workloads
“An IaaS cloud is a
highly automated
virtual infrastructure
that enables selfservice resource
requests, and
consumption of the
shared environment
is tracked for either
chargeback or
showback
purposes.”
Forrester Research
100’s of pilots and few production deployments in 2011; expected to be 10 times more in 2012 - Gartner
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

3. Capital Leverage
Workforce Leverage
Enterprise Objectives for Cloud
Self Service
Remove IT as a service delivery critical path
Management
Automation
Reduce IT operational costs
Workload
Standardization
Consistent application and service deployment
Usage Metering
Visibility into user and line of business usage
Centralized
Management
Smarter Virtualization
Manage complete infrastructure, regardless of scale
Drive reduced capital requirements
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

4. Traditional Data Center
Amazon-style Cloud
Legacy Availability Zone
CloudStack Management Server
vCenter
vSphere
AND
OR
Enterprise Networking (e.g., VLAN)
ESXi
Cluster
ESXi
Cluster
Availability
Zone
Availability
Zone
ESXi
Cluster
Enterprise Storage (e.g., SAN)
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy
Object Storage
Availability
Zone

5. Best practices aren’t always

6. Density in the cloud

7. Traditional Server Virtualization
• Core Objectives
ᵒServer consolidation
ᵒPower and cooling savings
ᵒHardware independence
• Looks Like
ᵒVM Density < 20
ᵒvCPU = pCPU
ᵒvRAM = pRAM
ᵒLow IOPS
ᵒRedundancy matters
ᵒNo templates
7
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

8. Desktop Virtualization
• Core Objectives
ᵒControl of IP
ᵒEnsuring patch compliance
ᵒSupporting mobile workstyles
• Looks Like
ᵒ50 -100 VMs per host
ᵒ2-4 vCores = pCore
ᵒ1-2 vRAM = pRAM
ᵒHigh IOPS
ᵒBoot storms
ᵒNetwork contention
ᵒHighly templated
8
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

9. Cloud Services
• Core Objectives
ᵒAgile provisioning
ᵒHigh degrees of tenant isolation
ᵒLow operating margins
• Looks Like
ᵒ50-250 VMs per host
ᵒ2-8 vCore = pCore
ᵒvRAM = pRAM
ᵒModerate IOPS
ᵒNetwork contention
ᵒLargely templated
9
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

10. Planning the network

11. Before Virtualization
• Simple management model
• Provisioning took a long time
• Topologies fairly static
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

12. Along Comes Server Virtualization
• Multiple VMs/host
ᵒLoss of visibility
ᵒLoss of control
• Edge moves into host
ᵒNetwork admins need to understand
server virtualization
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

13. Example 1 – Mirroring Traffic
• Without virtualization this is pretty
easy
• With virtualization you now have
multiple VMs
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

14. Example 1 – Mirroring Traffic
• Without virtualization this is pretty
easy
• With virtualization you now have
multiple VMs
ᵒPlus VMs can move
• Better to monitor at virtual switch
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

15. Example 2 – Network Policies
• Server admins have significant impact
on the network
ᵒIP and MAC Address
ᵒVirtual NICs
ᵒProtocols and ports
• Granular network control requires
awareness of virtual machines
ᵒDefine policies at virtual switch
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

16. Network Management Tools Lag
• Assumptions of fixed topology
ᵒFine for physical
ᵒChallenge for dynamic environment
• Not virtualization aware
ᵒIncorrect topology
ᵒIncomplete topology
ᵒVM actions obsolete data
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy
X

17. Virtual Machine Density Planning
• Host capacities are growing rapidly
ᵒvSphere 5 > 512 VMs
ᵒRHEV 3 > 1000 VMs
ᵒHyper-V > 2048 VMs
• Clouds and VDI push limits
• Top of rack switch selection matters?
ᵒARP table
ᵒSwitching performance drops
ᵒVM starts, but can’t connect
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy
Host 2
VM
VM
VM
VM
VM
VM
VM
VM
VM
Host 1
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM

18. Storage choice is critical

19. Shared storage growth and provisioning time
VMs
VMs
500
1,000
500
100
200
Cost, AU
AU – arbitrary units
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy
Provisioning efficiency

20. Combined efficiency and storage evolution
VMs
VMs
Redesign
1,000
500
500
?
1,000
100
200
Cost, AU
AU – arbitrary units
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy
Alternatives
100
200
Cost, AU

21. Efficiency and pod storage
No redesign
VMs
VMs
Redesign
1,000
POD #3
1,000
POD #2
500
500
POD #1
100
200
Cost, AU
AU – arbitrary units
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy
100
200
Cost, AU

22. What about local storage?
VMs
VMs
50
1,000
500
100
200
Cost, AU
AU – arbitrary units
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy
Provisioning efficiency

23. Cost-Performance Trends
Local storage
trend
VMs
POD
trend
VMs
1,000
1,000
Traditional
trend
500
500
Performance
trend
Local storage
100
200
Cost, AU
Shared Storage
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy
100
200
Local Storage
Cost, AU

24. Understanding disk usage and sizing
VM_DISK OS_PARTITION
USR_DATA
SWAP
TOTAL_DISK
VM_COUNT * VM_DISK + SWAP = TOTAL_DISK
VM_COUNT * (OS_PARTITION + USR_DATA) + SWAP = TOTAL_DISK
VM_COUNT = (TOTAL_DISK – SWAP) ÷ (OS_PARTITION + USR_DATA)
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

25. Templates and thin provisioning matter
USR_DATA
OS_PARTITION
SWAP
TOTAL_DISK
VM_COUNT * USR_DATA + OS_PARTITION + SWAP = TOTAL_DISK
VM_COUNT = (TOTAL_DISK – SWAP – OS_PARTITION) ÷ USR_DATA
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

26. Storage performance
IO per Disk
Write Penalties
RAID
PENALTY
VM Utilization
RPM
IOPS
ITEM
~VALUE
SSD
5,000+
0
1
IOPS per VM
20
SAS 15,000
175
1
2
Size, KB
4-8
SAS 10,000
125
5
4
Writes, %
80
SAS 7,200
75
6
6
Reads, %
20
10
2
50
4
IOPS = [IOPS per DISK]*[Disk Count]*([% of Reads]+[% of Writes] ÷ [RAID Write Penalty])
VM_COUNT = IOPS ÷ [IOPS per VM]
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

27. Blueprint for success ….

28. Cloud Builder Lessons from Zynga
• Public clouds are minivans
• zCloud is a race car
ᵒzCloud is optimized for social gaming
ᵒKnow your application requirements
• Don’t rent what you can own cheaper
ᵒCloud operator doesn’t care about your success
ᵒOptimized applications might be key
• Ensure you have backup plans
ᵒUsage can and does spike
ᵒOutages can and do happen
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy
vs.

29. Cloud Builder Lessons From Telcos
• Utility computing fits business model
ᵒTraditionally operate a low margin business model
ᵒUnderstand tiered service offerings
ᵒHave a history with instant provisioning
• Tiered service demands infrastructure flexibility
ᵒ“Cost per instance” is paramount
ᵒCharge extra for premium features
ᵒInstance doesn’t imply virtualization
ᵒBe prepared to change vendors if better model appears
• Provisioning agility expected
ᵒCustomers expect instant self service access and detailed billing
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

30. Service Offerings
• Clearly define what you want to offer
ᵒWhat types of applications
ᵒWho has access, and who owns them
ᵒWhat type of access
• Define how templates need to be managed
ᵒOperating system support
ᵒPatching requirements
• Define expectations around compliance and availability
ᵒWho owns backup and monitoring
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

31. Define Tenancy Requirements
• Department data local to department
ᵒWhere is the application data stored
• Data and service isolation
ᵒVM migration and host HA
ᵒNetwork services
• Encryption of PII/PCI
ᵒWhere do keys live when data location unknown
ᵒNeed encryption designed for the cloud
• Showback to stakeholders
ᵒMore than just usage, compliance and audits
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

32. Virtualization Infrastructure
• Hypervisor defined by service offerings
ᵒDon’t select hypervisor based on “standards”
ᵒUnderstand true costs of virtualization
ᵒMultiple hypervisors are “OK”
ᵒBare metal can be a hypervisor
• To “Pool” resources or not
ᵒIs there a real requirement for pooled resources
ᵒCan the cloud management solution do better?
ᵒReal cost of shared storage
• Primary storage defined by hypervisor
• Template storage defined by solution
ᵒTypically low cost options like NFS
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

33. Cloud Operations
• Design for maintainability
• Monitor critical components
ᵒManagement servers and system support VMs
ᵒHypervisor hosts, and critical infrastructure
ᵒEnd user deployment environments
If your cloud has maintenance windows, you’re doing it wrong.
- Allan Leinwand Former CTO Zynga
© Citrix 2013. More information at xenserver.org and follow me on twitter @XenServerArmy

34. Work better. Live better.