Hello my name isI amDatapipe isWe’ve built a large production global cloudstack cloud that is available in public, hybrid, and private flavors.Today I’m going to try to Explore the bits and pieces around a production cloudstack deployment. We only have 45 minutes, so I’ll be covereing some of these topics at a surface level. Would be great if this could be interactive so I’ll pause for questions throughout the presentaiton.
5 Regions10+ Zones10GECloudStack 2.2.14GA July 2012Time to beta – 6 months
N+NReliable power and coolingRoom for growthRemote Hands Support or Managed Service capability
Lots of concurrent VLAN support, hypervisor awareness, and programmability.
So much opportunity for software developers in these spaces.
Solving the CloudStack puzzle. The complete stack explored.
DISCLAIMER • No endorsements of
products by Datapipe implied. • Any opinions are my own. • YMMV. Use this information at your own risk. • Thanks to reddit for the pics • Enjoy!
CloudStack: The Key Ingredient.• Strong
open source community• Works at scale in production today, yet easy to get started• Wide ISV and ecosystem support• Compatible will all major computing, network, storage, and hypervisor options, including AWS• Lots of room for DIFFERENTIATION
But you need moreTo deploy
at scale You need: A whole bunch of technology assets andTo deploy production software.workloads – public orprivate Great people and expertise. Willingness to experiment, learn, and grow
You Need a Datacenter* Three
ways to get one: 1. Have one already 2. Build one (expensive)3. Rent space in one (cheaper)* This is where many folks stopand use someone else’s cloud.
CloudStack requires a reliable and
flexible network design.• 10GbE• Redundant links to each component• Separate management interface for network KVM• Look for broad standards support, ability to deploy in small, reliable chunks, loads of bandwidth. You’ll need that for your IP based storage.• Doesn’t need to be too fancy. But needs to be PLANNED.• CloudStack will handle handle much of your tenant networking. “Pseudo-SDN. “
Security• Programmable routing and firewall
rules are really important. You’ll want flexibility.• Ability to design network separation between management, guest, and utility networks.• Juniper SRX works well with CloudStack. Well known, lots of support, lots of different models available.• Nail down your CloudStack network model early. Rebuilding Zones and VLANs, IP bindings not fun.
Computing• Buy servers.• Lots of
RAM.• Reliability does matter. Be careful with off- brands.• We like simple 1U or 2U pizza boxes that can be ripped and replaced. YMMV.
Storage (aka My Precious)• We
use a variety of vendors and storage designs to surface reliable PRIMARY and SECONDARY storage.• We chose to standardize on NFS for both storage models, across hypervisors. Keep it simple.• Use different storage for management (Murphy’s Law)• IP based. Use those 10GbE pipes.• CloudStack doesn’t include Object Storage.
Dashboard & Portal• What your
customers see. The surface of your cloud.• Build your own? Need talented software devs. You can respond to customers quickly.• OR Use commercial product• OR keep it simple with basic CloudStack UI and server automation scripting. Depends on your use case.
Logging• CloudStack has limited logging
capability; not very useful for operational management.• Need to alert on specific warnings, analyze and pinpoint issues, etc.• Splunk is your friend.• Aggregate usage data as well - report on templates, Service Offerings, etc
Monitoring• Cloud Ops requires extensive
monitoring at all levels, from physical infrastructure and system resources, through the hypervisors, to the guest resources.• Built in monitoring is limited to non-existent.• We use a mix of Zenoss and some proprietary IP.• Why Zenoss? It’s CloudStack compatible, agentless, and has a decent API to build on.• We surface this as “Advanced Monitoring” via our portals and API endpoints. Think about how you offer monitoring.
Metering• CloudStack has limited metering
for certain network models• Third-party products can be plugged in: – InMon Traffic Sentinel• You still need to build your own metering for any differentiated offerings.• Metering needs to tie into your billing and commercial model
Reporting and Billing• Users and
business execs want reports.• Users need to pay or allocate funds for chargeback.• CloudStack has very limited reporting and billing capabilities.• With combination of direct database access, a logging tool, and some API calls, very elegant reporting and billing interfaces can be built.• We aggregate data in a NoSQL DB cluster (REDIS) for quick and instant reporting and reconciliations.
• Great API. But ….
The default URL endpoint is ugly, insecure, inflexible, and not highly available.http://somecloudstackmanagemen tserver:8080/client/api• HAProxy and nginx load balancers/reverse proxy architecure. SSL encryption.https://cloud.datapipe.com/api/c ompute/v1 –Pretty. Secure. Flexible. Reliable.
Even more stuff• Documentation• Hypervisor
Management• Support and Ticketing• Server Automation• Template Management• Capacity Management• Sales force automation/CRM integration• Marketing automation integration• Guest Security, VPN, and Load Balancing• Application Stacks• …..and the list goes on.