June 2014 HUG - Continuuity Loom : Cluster Management
100% Open Source, multi-tiered application
infrastructure provisioning and management!
Nitin Motgi Albert Shau
• Create variety of application infrastructure stack - Continuuity Reactor + Hadoop
(CDH, HDP, Apache), Secured Hadoop, LAMP, Mesos, …
• In Public and Private Cloud
• Using our existing investment in SCM tools - Chef, Shell script, etc
• Easy way to manage the stack - like add new services, manage conﬁguration
changes, scale up/down, etc..
• DevOps friendly, easy to manage and maintain
• Resource constraints (1 Ninja support 20 Developers) - Ninja had no vacations or
couldn’t get sick
Multi-tier applications are hard to provision, install, conﬁgure, monitor & scale and also have
complex cluster life cycles.
Lack of experience and repeatability
Installation and Conﬁguration require a lot of practice to tackle common pitfalls for conﬁguring complex
multi-tier application clusters, complexity prevents simple attempts at automation
Increased organization bottlenecks
Complexity limits number of people who can provision, install and conﬁgure complex multi-tier
Decreased developer productivity
Hard to support DevOps role within an organization without an easy to use self serving model
No governance with reference architectures
Difﬁcult to model reference architectures in complex environments
Multiple cloud environments
Supporting multiple cloud environments is extremely hard.
• Numerous technologies available that solve parts of the
• Open Source / Closed Source
• We looked closely at Apache Ambari, Crowbar, etc…
• Concluded that none at current state were solving our
problem (Happy to talk about it after this talk)
A system for templatizing and materializing complex multi-tiered application
reference architectures in public or private clouds. Designed bottom-up to
support different facets of your organization - from developers, operations and
system administrators to large service providers.
Who can use it ?
TEMPLATES ON CLOUD
Create, provision and
of complex, multi-tiered
applications using a
Write Chef recipes,
and shell scripts for
Create and manage
Continuuity Loom templates
for multi-tiered application
compliance policies and
enforce service constraints.
Instance of !
on any cloud
• Software component. ex: Namenode, Datanode, ResourceManager, etc.
• Specify dependencies between services
• Deﬁnes how a cluster should be created
• Specify compatible services, conﬁguration and constraints which determine service placement
• Creates and destroys machines. ex: AWS, Rackspace, Joyent, Openstack
• Performs service actions like start, stop, conﬁg, install, remove, init
• ex: chef, puppet, scripts, etc.