Future of Apache CloudStack: modular,
distributed, and hackable
Darren Shepherd
Citrix

CCCEU 2013
About Me
●

Apache CloudStack Committer

●

Previously - Cloud Architect @ GoDaddy.com
–

Built three generations of cloud...
What's This About
●

My opinions

●

What I think we should focus on architecturally

●

What I'm trying to accomplish
General Comments
●

Do more with less
–

Identify patterns in code and create frameworks
●

–
●

For example, make Abstrac...
Modular – Why?
●

Enforce design and architectural boundaries

●

Better pluggability

●

Quality/Testing

●

Lays the fou...
Modular – What's been done
●

4.3 - Spring Modularization
–

META-INF/cloudstack/mycomponent/module.properties

–

META-IN...
Stats
●

~400k lines of Java (not 1.2m)
–
–

32% - engine, framework, services, usage, util

–
●

42% - agent, api, core, ...
What needs to be done
●

Package plug-ins in separate RPM/DEBs

●

Break out more code from /server/src

●

Break apart *M...
Distributed Architecture
●

Maybe you don't care
–

Many don't – Monolithic deployment is simple and
scalable enough for m...
Monolithic
Single JVM

Redundant JVM
Fully Distributed
●

System Services
–
–

●

Event Bus
Lock Manager

Servers
–

API

–

Back end Jobs

–

Host Manager
How
●

Event Bus
–

Hazelcast or Redis

–

Do not use “reliable messaging”
●

●

Lock Manager
–

●

Works more like UDP mu...
Hackability
●

Should be relatively simple to add one off
functionality
–

●

●

Every cloud deployment is slightly differ...
Totally scientific observation
●

Two types of sys admins
–

Those who hate Java for no good reason

–

Those who hate Jav...
Dynamic languages can be good

Those that
Those who
will tolerate developing
love systems
in a dynamic
(admins/engineers)
...
Hackability
●

Move systems related code to python

●

No Java on the hypervisor!

●

Need a new ServerResource architectu...
Questions?
Upcoming SlideShare
Loading in …5
×

Future of Apache CloudStack: modular, distributed, and hackable

1,300 views

Published on

CloudStack has been often critiqued because of its architecture. This session is to talk about the efforts that have been done and are in progress to move CloudStack's architecture forward. How we can achieve the simplicity of a monolithic system, but scalability of a distributed architecture. How we can keep the speed and reliability of Java, but leverage scripting languages for easier integrations.
How can we get CloudStack to be the best and easiest system to deploy and maintain for clouds from 10s to 100s of thousands of hypervisors.

Published in: Technology, Education
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,300
On SlideShare
0
From Embeds
0
Number of Embeds
44
Actions
Shares
0
Downloads
13
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Future of Apache CloudStack: modular, distributed, and hackable

  1. 1. Future of Apache CloudStack: modular, distributed, and hackable Darren Shepherd Citrix CCCEU 2013
  2. 2. About Me ● Apache CloudStack Committer ● Previously - Cloud Architect @ GoDaddy.com – Built three generations of clouds ● ● ● 2nd – Custom “CloudStack Compatible” ● – 1st – CloudStack 2.x, custom storage framework 3rd – 100% from scratch Been developing IaaS orchestration systems full time for the last 4 years Currently – Do something @ Citrix – Focus on open source – I care about architecture and design
  3. 3. What's This About ● My opinions ● What I think we should focus on architecturally ● What I'm trying to accomplish
  4. 4. General Comments ● Do more with less – Identify patterns in code and create frameworks ● – ● For example, make Abstract base classes Copy and paste is bad Need to leverage more open source frameworks – jOOQ, Spring TX, Netflix Archaius/Servo, Jackson, Apache jclouds, Flyway/Liquibase, Apache Commons ● Past/Current – ACS adds cloudiness to legacy technologies ● Current/Future – ACS orchestrates cloudy technologies ● Always provide a way for Apache CloudStack to do nothing and delegate to something else
  5. 5. Modular – Why? ● Enforce design and architectural boundaries ● Better pluggability ● Quality/Testing ● Lays the foundation for distributed architecture
  6. 6. Modular – What's been done ● 4.3 - Spring Modularization – META-INF/cloudstack/mycomponent/module.properties – META-INF/cloudstack/mycomponent/*context.xml – Over 50 Modules
  7. 7. Stats ● ~400k lines of Java (not 1.2m) – – 32% - engine, framework, services, usage, util – ● 42% - agent, api, core, server 26% - plugins We still have big blobs of code – 74% “Core Code” “Core Code” is not really core. Includes VR, CP, SSVM, XenServer, others
  8. 8. What needs to be done ● Package plug-ins in separate RPM/DEBs ● Break out more code from /server/src ● Break apart *ManagerImpl classes – ● Implement too many interfaces that belong in different architectural layers Modularize VR and VPC code
  9. 9. Distributed Architecture ● Maybe you don't care – Many don't – Monolithic deployment is simple and scalable enough for most use case ● Support multiple deployment models ● Scalability – ● Scale tiers according to load characteristics Reliability – Put certain components closer to resources
  10. 10. Monolithic Single JVM Redundant JVM
  11. 11. Fully Distributed ● System Services – – ● Event Bus Lock Manager Servers – API – Back end Jobs – Host Manager
  12. 12. How ● Event Bus – Hazelcast or Redis – Do not use “reliable messaging” ● ● Lock Manager – ● Works more like UDP multicast Hazelcast or ZooKeeper Same binary on all servers – “Profiles” used to determine which Spring modules are activated on start
  13. 13. Hackability ● Should be relatively simple to add one off functionality – ● ● Every cloud deployment is slightly different Need to stay relevant with upcoming technologies Want people to integrate first with ACS because – It works – It's easy
  14. 14. Totally scientific observation ● Two types of sys admins – Those who hate Java for no good reason – Those who hate Java for a good reason Those that will tolerate developing in Java Those who love systems (admins/engineers)
  15. 15. Dynamic languages can be good Those that Those who will tolerate developing love systems in a dynamic (admins/engineers) language
  16. 16. Hackability ● Move systems related code to python ● No Java on the hypervisor! ● Need a new ServerResource architecture that delegates to agents in any language – Work for Hyper-V is a step in the right direction ● Decouple VR from HypervisorResource ● Community help needed – Good work, GCE API and hopefully new AWS API
  17. 17. Questions?

×