0
100% Open Source, multi-tiered application
infrastructure provisioning and management!
Nitin Motgi Albert Shau
@nmotgi @al...
Our Problem
2
• Create variety of application infrastructure stack - Continuuity Reactor + Hadoop
(CDH, HDP, Apache), Secu...
3
Challenges
Simply Hard
Multi-tier applications are hard to provision, install, configure, monitor & scale and also have
c...
Our Assessment
4
• Numerous technologies available that solve parts of the
problem
• Open Source / Closed Source
• We look...
Introducing
5
A system for templatizing and materializing complex multi-tiered application
reference architectures in publ...
Who can use it ?
6
CONTINUUITY LOOM
TEMPLATES ON CLOUD
DEVELOPER
Create, provision and
decommission clusters
of complex, m...
Lifecycle
7
Create!
Loom!
Template
Save!
Template
Publish!
Template
Expose!
REST API
Create!
Instance of !
template
on any...
Concepts
8
• Service!
• Software component. ex: Namenode, Datanode, ResourceManager, etc.
• Specify dependencies between s...
Architecture
9
Cluster Creation: Solver
10
Template
+
args (size, etc.)
Solver
Layout
services:{s1, s3}
hw: large
image: ubuntu12
x 1
ser...
Cluster Creation: Planner
11
Layout Dependencies
Planner
Coordinator
Task Queue
ProvisionersProvisionersProvisionersProvis...
Planner : DAG
12
Planner : Linearize
13
Planner: Expand
14
DEMO
15
16
Thank you
Nitin Motgi Albert Shau
@nmotgi @albertshau
http://github.com/continuuity/loom
Upcoming SlideShare
Loading in...5
×

June 2014 HUG - Continuuity Loom : Cluster Management

491

Published on

June 2014 HUG - Continuuity Loom : Cluster Management

Published in: Software, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
491
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "June 2014 HUG - Continuuity Loom : Cluster Management"

  1. 1. 100% Open Source, multi-tiered application infrastructure provisioning and management! Nitin Motgi Albert Shau @nmotgi @albertshau tryloom.io
  2. 2. Our Problem 2 • Create variety of application infrastructure stack - Continuuity Reactor + Hadoop (CDH, HDP, Apache), Secured Hadoop, LAMP, Mesos, … • In Public and Private Cloud • Using our existing investment in SCM tools - Chef, Shell script, etc • Easy way to manage the stack - like add new services, manage configuration changes, scale up/down, etc.. • DevOps friendly, easy to manage and maintain • Resource constraints (1 Ninja support 20 Developers) - Ninja had no vacations or couldn’t get sick
  3. 3. 3 Challenges Simply Hard Multi-tier applications are hard to provision, install, configure, monitor & scale and also have complex cluster life cycles. Lack of experience and repeatability Installation and Configuration require a lot of practice to tackle common pitfalls for configuring complex multi-tier application clusters, complexity prevents simple attempts at automation Increased organization bottlenecks Complexity limits number of people who can provision, install and configure complex multi-tier application clusters Decreased developer productivity Hard to support DevOps role within an organization without an easy to use self serving model No governance with reference architectures Difficult to model reference architectures in complex environments Multiple cloud environments Supporting multiple cloud environments is extremely hard.
  4. 4. Our Assessment 4 • Numerous technologies available that solve parts of the problem • Open Source / Closed Source • We looked closely at Apache Ambari, Crowbar, etc… • Concluded that none at current state were solving our problem (Happy to talk about it after this talk)
  5. 5. Introducing 5 A system for templatizing and materializing complex multi-tiered application reference architectures in public or private clouds. Designed bottom-up to support different facets of your organization - from developers, operations and system administrators to large service providers.
  6. 6. Who can use it ? 6 CONTINUUITY LOOM TEMPLATES ON CLOUD DEVELOPER Create, provision and decommission clusters of complex, multi-tiered applications using a self-service model. SYS ADMIN Write Chef recipes, Puppet modules and shell scripts for application services. IT OPERATIONS Create and manage Continuuity Loom templates for multi-tiered application services. Configure compliance policies and enforce service constraints.
  7. 7. Lifecycle 7 Create! Loom! Template Save! Template Publish! Template Expose! REST API Create! Instance of ! template on any cloud Reference ! Architecture! (Enterprise) Centralized! Administration! Control with! Multi-tenancy Supports Hybrid! Infrastructure! & Architecture
  8. 8. Concepts 8 • Service! • Software component. ex: Namenode, Datanode, ResourceManager, etc. • Specify dependencies between services • Template! • Defines how a cluster should be created • Specify compatible services, configuration and constraints which determine service placement • Provider! • Creates and destroys machines. ex: AWS, Rackspace, Joyent, Openstack • Automator! • Performs service actions like start, stop, config, install, remove, init • ex: chef, puppet, scripts, etc.
  9. 9. Architecture 9
  10. 10. Cluster Creation: Solver 10 Template + args (size, etc.) Solver Layout services:{s1, s3} hw: large image: ubuntu12 x 1 services:{s2} hw: medium image: ubuntu12 x 4
  11. 11. Cluster Creation: Planner 11 Layout Dependencies Planner Coordinator Task Queue ProvisionersProvisionersProvisionersProvisionersProvisionersProvisioners Plan
  12. 12. Planner : DAG 12
  13. 13. Planner : Linearize 13
  14. 14. Planner: Expand 14
  15. 15. DEMO 15
  16. 16. 16 Thank you Nitin Motgi Albert Shau @nmotgi @albertshau http://github.com/continuuity/loom
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×