Future of Apache CloudStack: modular, distributed, and hackable

Future of Apache CloudStack: modular,
distributed, and hackable
Darren Shepherd
Citrix

CCCEU 2013

About Me
●

Apache CloudStack Committer

●

Previously - Cloud Architect @ GoDaddy.com
–

Built three generations of clouds
●
●

●

2nd – Custom “CloudStack Compatible”

●

–

1st – CloudStack 2.x, custom storage framework
3rd – 100% from scratch

Been developing IaaS orchestration systems full time for the last 4
years

Currently – Do something @ Citrix
–

Focus on open source

–

I care about architecture and design

What's This About
●

My opinions

●

What I think we should focus on architecturally

●

What I'm trying to accomplish

General Comments
●

Do more with less
–

Identify patterns in code and create frameworks
●

–
●

For example, make Abstract base classes

Copy and paste is bad

Need to leverage more open source frameworks
–

jOOQ, Spring TX, Netflix Archaius/Servo, Jackson, Apache jclouds,
Flyway/Liquibase, Apache Commons

●

Past/Current – ACS adds cloudiness to legacy technologies

●

Current/Future – ACS orchestrates cloudy technologies

●

Always provide a way for Apache CloudStack to do nothing and
delegate to something else

Modular – Why?
●

Enforce design and architectural boundaries

●

Better pluggability

●

Quality/Testing

●

Lays the foundation for distributed architecture

Modular – What's been done
●

4.3 - Spring Modularization
–

META-INF/cloudstack/mycomponent/module.properties

–

META-INF/cloudstack/mycomponent/*context.xml

–

Over 50 Modules

Stats
●

~400k lines of Java (not 1.2m)
–
–

32% - engine, framework, services, usage, util

–
●

42% - agent, api, core, server
26% - plugins

We still have big blobs of code
–

74% “Core Code”

“Core Code” is not really core. Includes VR, CP,
SSVM, XenServer, others

What needs to be done
●

Package plug-ins in separate RPM/DEBs

●

Break out more code from /server/src

●

Break apart *ManagerImpl classes
–

●

Implement too many interfaces that belong in
different architectural layers

Modularize VR and VPC code

Distributed Architecture
●

Maybe you don't care
–

Many don't – Monolithic deployment is simple and
scalable enough for most use case

●

Support multiple deployment models

●

Scalability
–

●

Scale tiers according to load characteristics

Reliability
–

Put certain components closer to resources

Monolithic
Single JVM

Redundant JVM

Fully Distributed
●

System Services
–
–

●

Event Bus
Lock Manager

Servers
–

API

–

Back end Jobs

–

Host Manager

How
●

Event Bus
–

Hazelcast or Redis

–

Do not use “reliable messaging”
●

●

Lock Manager
–

●

Works more like UDP multicast

Hazelcast or ZooKeeper

Same binary on all servers
–

“Profiles” used to determine which Spring modules are
activated on start

Hackability
●

Should be relatively simple to add one off
functionality
–

●

●

Every cloud deployment is slightly different

Need to stay relevant with upcoming
technologies
Want people to integrate first with ACS because
–

It works

–

It's easy

Totally scientific observation
●

Two types of sys admins
–

Those who hate Java for no good reason

–

Those who hate Java for a good reason

Those that
will tolerate developing
in Java

Those who
love systems
(admins/engineers)

Dynamic languages can be good

Those that
Those who
will tolerate developing
love systems
in a dynamic
(admins/engineers)
language

Hackability
●

Move systems related code to python

●

No Java on the hypervisor!

●

Need a new ServerResource architecture that
delegates to agents in any language
–

Work for Hyper-V is a step in the right direction

●

Decouple VR from HypervisorResource

●

Community help needed
–

Good work, GCE API and hopefully new AWS API

Future of Apache CloudStack: modular, distributed, and hackable

More Related Content

What's hot

Similar to Future of Apache CloudStack: modular, distributed, and hackable

Recently uploaded

Future of Apache CloudStack: modular, distributed, and hackable