Open source Cloud Automation Platform

http://www.gomegam.com
Megam : Under the Hood
From IaaS to PaaS ? Are you ready ?

© 2014 Megam Systems
Why PaaS ?
What Are My Options ?
How You Can Build One.
Architecture/Know How of Megam
Our Working Prototype – Screencast #1

Why PaaS
Business needs agility with faster time to market
Cycle time to build apps can be shortened ?
Cloud Apps on Hybrid cloud
Apps on single cloud are fragile
Businesses need 24 x 7 uptime
Hybrid Scaling, DR of Cloud Apps is a pain

Enterprise IT Migration cycle
SaaS
PaaS
IaaS
Rehost
Refactor
Revise
Rebuild
Replace

The market will experience consistent growth with
worldwide PaaS revenue totaling 1.5 billion in 2013, and
growing to $2.9 billion in 2016
The SaaS-based cloud market will grow from $12.1B in
2013 to $21.3B in 2015, with the primary growth factors
being ease of customization and speed of deployment.
Gartner

What Are My Options ?

Megam - Features
➔
Write Once, Run on Any Cloud.
➔
Apps Marketplace – Easy to consume canned apps.
➔
Cloud in a box for Bare metal
➔
Log Analytics, Server/Application Monitoring
➔
For more info: visit http://www.gomegam.com
Target Market
Data centers, Enterprise, SMBs, DR vendors, SCM vendors, Developers, Training,
Software makers

Definition driven architecture
➔
describe dependencies to move to cloud.

What to describe ?
Describe the VM which stands up an App/DB (or) Scaling,
Loadbalancing, DR

For simplicity assume a
Ruby on Rails App

We need a Medium to describe
➔
WebApp, CLI, Mob

In the Medium We describe a
Cloud named sandbox_default

We created a cloud named
sandy_default
Let us use it

In the Medium we describe a
app by choosing a framework

To stand that app
in cloud : sandbox_default

You Can Watch
Logs real time/ off-line

Manage from a Dashboard

Enough of Talking
Show me the Use case

On board
www.megam.co
Start
Create App
Choose
Java App
Point Git
or WAR
Choose
Cloud
Launch App
Select
Logs
Watch
Real-time Log
Select
Monitor App
Monitor/
Watch App Metric
..API
Server
Herk

What more can I do ?
Contact Us : gomegam@megam.co.in
Phone: +613-834-4518
Guide: http://www.gomegam.com/docs
Open source Apache V2.

What happens Internally ?
Sure
(Rest of the slides has in depth design)

What happens during description of an app
A node is a top level description of a cloud resource (app,
services).
<_>Defns
Canhave

A <_>Defn is a description of a cloud resource by specifying its
constraints and characteristics.

App Defn:
A Web App or mob HTML5
or API Service
Nut Defn
A Group composed of Appn
+
Wrenchn
+ Boltn
Wrench Defn
Composes App Defns (eg:
HA Proxy)
Bolt Defn
Service: Attaches to an App
(eg: DB, Queue
SSL..)
4 types of Defn

Scaling app is easy.
app can be combined with bolt and viceversa.
If more than one app is requested, then you need a wrench to compose them.
||
=
=
+
In this case 2 independent apps will be
created

node =>
➔ Has a type, name, domain
➔ With predef (flavor of framework : java, scala, go, <?>...)
➔ Has required runtime dependencies: scm
➔ Has an exec context
➔ Can have many requests based on req_type

node Has
Sub group
Termed Predef
Sub category
Termed Requests
RoR
Ruby
Go
Erlang
Scala
Play
Java
Riak
PostgreSQL
RabbitMQ
Akka
...
CREATE
NUKE
STOP
NSTOP
NSTART
NRESTART
App
Bolt
Wrench
Nut

Watch our screencast #1

Data structure
Orgs
Users
Cloud
Apps
Products
prod_id
cloud_identity_id
users_id
org_id
api_key
accounts nodes predefs
Predef
clouds
/1(org_id)
/2(org_id)
Cloud Books
Histories
Cloud Statistics
Cloud Books
requests
Dashboards
Widgets

Decomposing● Web App
– nilavu
● API Server
– megam_play
● Cloud Herk
– megam_akka
● Gulp
– gulp
● Cloud Tool
– megam_chef
● Common libs
– megam_common
● Ruby API
– megam_api
● Cloud Real time
– tap
● CLI
– meggy
Web App CLI
API Server
Herk Common libs
Ruby APICloud Real time
Cloud Tool
Mob
Gulp

WebApp
● Ruby on Rails
● Customers will login to https://www.megam.co
●
Create an account
– Generate a a shared api_key.
● Create Apps for free
●
Use default in built cloud settings (EC2).
● Metrics
– Meter and Monitor cloud books
– Off-line/Real time log
● Dash – Built using AngularJS
●
https://github.com/megamsys/nilavu.git
● Status : 0.3 – beta

API Server
● Stateless RESTful API Server
● Functional Programming (Scala), Scalaz 7
● HMAC based Auth.
● Protected resources
● https://api.megam.co
● https://github.com/megamsys/megam_play.git
● Status : 0.1 – beta

Command Line(CLI)
● CLI built using mixlib::cli (ruby), follows the
chef cli style.
● Commands
– account
– nodes (CRUD)
– logs
● Status : 0.1 – alpha
● https://github.com/indykish/meggy

Cloud Real-Time Streamer
● Real-time streaming server
● Clustered to the number of cores, restarts itself on
failure.
● Connects to Redis
● Uses express/socket.io
● Stream content based on URL /streams/:id {where
id = a node}
● https://github.com/megamsys/tap.git
● Status : 0.3 beta

Redis
Tap Monitor
Tap Cluster
Tap 0
Tap 1
Tap n
Redis
Tap Cluster
Cluster current If fail over
n based on core
Current If Fail Over
Beaver
Agent
Beaver
Agent
Beaver
Agent
Beaver
Agent
v1/nodes/logs/:id
/node0
User clicks on an
URL
/node1
/noden
(or) requests log
from pug
n1
nn
n2

Cloud Provisioner
● A facade provisioner which calls a cloud
management tool. Supported (Chef11)
● Support for hosted and private chef.
● Instrumented from Herk – Cloud Bridge
● https://github.com/megamsys/megam_chef.git

Cloud ProvisionerJSON
JSON
Validated
Async Future
Init
~/.megam/megam_chef.yaml
{
"systemprovider": {
"provider": {
"prov": "chef"
}
},
"compute": {
"ec2": {
"groups": "megam",
"image": "ami-56e6a404",
"flavor": "m1.small"
},
"access": {
"ssh-key": "megam_ec2",
"identity-file":
"~/.ssh/megam_ec2.pem",
"ssh-user": "ubuntu"
}
},
"chefservice": {
"chef": {
"command": "knife",
"plugin": "ec2 server create",
"run-list": "'role[opendj]'",
"name": "-N TestOverAll"
}
}
}
chef_repo
tested
cookbooks
knife-1
knife-2
knife-n
.
.
.
.
.
.

Herk – Cloud Bridge
● Main Clustered Akka
– Master which manages several slaves of cloud
node
– pubsub to RabbitMQ
● https://github.com/megamsys/megam_akka.git

Gulpd - Native
● Golang based native bridge
– Slave that is responsible to mutate a node
– pubsub to RabbitMQ
● https://github.com/megamsys/gulp.git

Router
Seed#1
Zk
/machines
/instances
/nodes
/pogo
...
vhost 'megam'
/megam_conf
/pogo(node#1)
…
/rogo(node#2)
Seed#2
Clo#1
Clo#2
Master
Worker#1
Master
Worker#2
Request
Dispatcher#1
Request
Dispatcher#3
Request
Dispatcher#2
Slave #1 Slave #3Slave #2
CloudProvisioner
Gulp
#1
Gulp
#1

How to contribute...
Step 1 : Sign our contributor's agreement:
Email: gomegam@megam.co.in
Step 2: We use Github https://github.com/megamsys
Step 3: Setup Dev Environment.
Step 4: Contact : gomegam@megam.co.in for our wishlist or
better yet suggest one.
Step 5. Join our forum
https://groups.google.com/forum/?fromgroups#!forum/megamlive

Open source Cloud Automation Platform

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Open source Cloud Automation Platform

Similar to Open source Cloud Automation Platform (20)

Recently uploaded

Recently uploaded (20)

Open source Cloud Automation Platform