Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
http://www.gomegam.com
Megam : Under the Hood
From IaaS to PaaS ? Are you ready ?
© 2014 Megam Systems
Why PaaS ?
What Are My Options ?
How You Can Build One.
Architecture/Know How of Megam
Our Working Pr...
© 2014 Megam Systems
Why PaaS
Business needs agility with faster time to market
Cycle time to build apps can be shortened ...
© 2014 Megam Systems
Enterprise IT Migration cycle
SaaS
PaaS
IaaS
Rehost
Refactor
Revise
Rebuild
Replace
© 2014 Megam Systems
The market will experience consistent growth with
worldwide PaaS revenue totaling 1.5 billion in 2013...
© 2014 Megam Systems
What Are My Options ?
© 2014 Megam Systems
Megam
© 2014 Megam Systems
Megam - Features
➔
Write Once, Run on Any Cloud.
➔
Apps Marketplace – Easy to consume canned apps.
➔
...
© 2014 Megam Systems
Definition driven architecture
➔
describe dependencies to move to cloud.
© 2014 Megam Systems
What to describe ?
Describe the VM which stands up an App/DB (or) Scaling,
Loadbalancing, DR
© 2014 Megam Systems
For simplicity assume a
Ruby on Rails App
© 2014 Megam Systems
We need a Medium to describe
➔
WebApp, CLI, Mob
© 2014 Megam Systems
In the Medium We describe a
Cloud named sandbox_default
© 2014 Megam Systems
We created a cloud named
sandy_default
Let us use it
© 2014 Megam Systems
In the Medium we describe a
app by choosing a framework
© 2014 Megam Systems
To stand that app
in cloud : sandbox_default
© 2014 Megam Systems
You Can Watch
Logs real time/ off-line
© 2014 Megam Systems
Manage from a Dashboard
© 2014 Megam Systems
Enough of Talking
Show me the Use case
© 2014 Megam Systems
On board
www.megam.co
Start
Create App
Choose
Java App
Point Git
or WAR
Choose
Cloud
Launch App
Selec...
© 2014 Megam Systems
What more can I do ?
Contact Us : gomegam@megam.co.in
Phone: +613-834-4518
Guide: http://www.gomegam....
© 2014 Megam Systems
What happens Internally ?
Sure
(Rest of the slides has in depth design)
© 2014 Megam Systems
What happens during description of an app
A node is a top level description of a cloud resource (app,...
© 2014 Megam Systems
A <_>Defn is a description of a cloud resource by specifying its
constraints and characteristics.
© 2014 Megam Systems
App Defn:
A Web App or mob HTML5
or API Service
Nut Defn
A Group composed of Appn
+
Wrenchn
+ Boltn
W...
© 2014 Megam Systems
Scaling app is easy.
app can be combined with bolt and viceversa.
If more than one app is requested, ...
© 2014 Megam Systems
node =>
➔ Has a type, name, domain
➔ With predef (flavor of framework : java, scala, go, <?>...)
➔ Ha...
© 2014 Megam Systems
node Has
Sub group
Termed Predef
Sub category
Termed Requests
RoR
Ruby
Go
Erlang
Scala
Play
Java
Riak...
© 2014 Megam Systems
Watch our screencast #1
© 2014 Megam Systems
Data structure
Orgs
Users
Cloud
Apps
Products
prod_id
cloud_identity_id
users_id
org_id
api_key
accou...
© 2014 Megam Systems
Decomposing● Web App
– nilavu
● API Server
– megam_play
● Cloud Herk
– megam_akka
● Gulp
– gulp
● Clo...
© 2014 Megam Systems
WebApp
● Ruby on Rails
● Customers will login to https://www.megam.co
●
Create an account
– Generate ...
© 2014 Megam Systems
API Server
● Stateless RESTful API Server
● Functional Programming (Scala), Scalaz 7
● HMAC based Aut...
© 2014 Megam Systems
Gateway
© 2014 Megam Systems
Command Line(CLI)
● CLI built using mixlib::cli (ruby), follows the
chef cli style.
● Commands
– acco...
© 2014 Megam Systems
Cloud Real-Time Streamer
● Real-time streaming server
● Clustered to the number of cores, restarts it...
© 2014 Megam Systems
Redis
Tap Monitor
Tap Cluster
Tap 0
Tap 1
Tap n
Redis
Tap Cluster
Cluster current If fail over
n base...
© 2014 Megam Systems
Cloud Provisioner
● A facade provisioner which calls a cloud
management tool. Supported (Chef11)
● Su...
© 2014 Megam Systems
Cloud ProvisionerJSON
JSON
Validated
Async Future
Init
~/.megam/megam_chef.yaml
{
"systemprovider": {...
© 2014 Megam Systems
Herk – Cloud Bridge
● Main Clustered Akka
– Master which manages several slaves of cloud
node
– pubsu...
© 2014 Megam Systems
Gulpd - Native
● Golang based native bridge
– Slave that is responsible to mutate a node
– pubsub to ...
© 2014 Megam Systems
Router
Seed#1
Zk
/machines
/instances
/nodes
/pogo
...
vhost 'megam'
/megam_conf
/pogo(node#1)
…
/rog...
© 2014 Megam Systems
How to contribute...
Step 1 : Sign our contributor's agreement:
Email: gomegam@megam.co.in
Step 2: We...
Upcoming SlideShare
Loading in …5
×

Open source Cloud Automation Platform

2,991 views

Published on

The cloud is becoming a medium. How do we make it more flatter and seamless. Cloud Apps deployed in enterprise needs hooks to play it like a puppet. Megam Cloud is a PaaS built from ground up on top of Chef using Scala, Chef, RabbitMQ, Riak, RoR and Go. https://www.megam.co, https://www.gomegam.com

Published in: Technology, Education
  • Be the first to comment

Open source Cloud Automation Platform

  1. 1. http://www.gomegam.com Megam : Under the Hood From IaaS to PaaS ? Are you ready ?
  2. 2. © 2014 Megam Systems Why PaaS ? What Are My Options ? How You Can Build One. Architecture/Know How of Megam Our Working Prototype – Screencast #1
  3. 3. © 2014 Megam Systems Why PaaS Business needs agility with faster time to market Cycle time to build apps can be shortened ? Cloud Apps on Hybrid cloud Apps on single cloud are fragile Businesses need 24 x 7 uptime Hybrid Scaling, DR of Cloud Apps is a pain
  4. 4. © 2014 Megam Systems Enterprise IT Migration cycle SaaS PaaS IaaS Rehost Refactor Revise Rebuild Replace
  5. 5. © 2014 Megam Systems The market will experience consistent growth with worldwide PaaS revenue totaling 1.5 billion in 2013, and growing to $2.9 billion in 2016 The SaaS-based cloud market will grow from $12.1B in 2013 to $21.3B in 2015, with the primary growth factors being ease of customization and speed of deployment. Gartner
  6. 6. © 2014 Megam Systems What Are My Options ?
  7. 7. © 2014 Megam Systems Megam
  8. 8. © 2014 Megam Systems Megam - Features ➔ Write Once, Run on Any Cloud. ➔ Apps Marketplace – Easy to consume canned apps. ➔ Cloud in a box for Bare metal ➔ Log Analytics, Server/Application Monitoring ➔ For more info: visit http://www.gomegam.com Target Market Data centers, Enterprise, SMBs, DR vendors, SCM vendors, Developers, Training, Software makers
  9. 9. © 2014 Megam Systems Definition driven architecture ➔ describe dependencies to move to cloud.
  10. 10. © 2014 Megam Systems What to describe ? Describe the VM which stands up an App/DB (or) Scaling, Loadbalancing, DR
  11. 11. © 2014 Megam Systems For simplicity assume a Ruby on Rails App
  12. 12. © 2014 Megam Systems We need a Medium to describe ➔ WebApp, CLI, Mob
  13. 13. © 2014 Megam Systems In the Medium We describe a Cloud named sandbox_default
  14. 14. © 2014 Megam Systems We created a cloud named sandy_default Let us use it
  15. 15. © 2014 Megam Systems In the Medium we describe a app by choosing a framework
  16. 16. © 2014 Megam Systems To stand that app in cloud : sandbox_default
  17. 17. © 2014 Megam Systems You Can Watch Logs real time/ off-line
  18. 18. © 2014 Megam Systems Manage from a Dashboard
  19. 19. © 2014 Megam Systems Enough of Talking Show me the Use case
  20. 20. © 2014 Megam Systems On board www.megam.co Start Create App Choose Java App Point Git or WAR Choose Cloud Launch App Select Logs Watch Real-time Log Select Monitor App Monitor/ Watch App Metric ..API Server Herk
  21. 21. © 2014 Megam Systems What more can I do ? Contact Us : gomegam@megam.co.in Phone: +613-834-4518 Guide: http://www.gomegam.com/docs Open source Apache V2.
  22. 22. © 2014 Megam Systems What happens Internally ? Sure (Rest of the slides has in depth design)
  23. 23. © 2014 Megam Systems What happens during description of an app A node is a top level description of a cloud resource (app, services). <_>Defns Canhave
  24. 24. © 2014 Megam Systems A <_>Defn is a description of a cloud resource by specifying its constraints and characteristics.
  25. 25. © 2014 Megam Systems App Defn: A Web App or mob HTML5 or API Service Nut Defn A Group composed of Appn + Wrenchn + Boltn Wrench Defn Composes App Defns (eg: HA Proxy) Bolt Defn Service: Attaches to an App (eg: DB, Queue SSL..) 4 types of Defn
  26. 26. © 2014 Megam Systems Scaling app is easy. app can be combined with bolt and viceversa. If more than one app is requested, then you need a wrench to compose them. || = = + In this case 2 independent apps will be created
  27. 27. © 2014 Megam Systems node => ➔ Has a type, name, domain ➔ With predef (flavor of framework : java, scala, go, <?>...) ➔ Has required runtime dependencies: scm ➔ Has an exec context ➔ Can have many requests based on req_type
  28. 28. © 2014 Megam Systems node Has Sub group Termed Predef Sub category Termed Requests RoR Ruby Go Erlang Scala Play Java Riak PostgreSQL RabbitMQ Akka ... CREATE NUKE STOP NSTOP NSTART NRESTART App Bolt Wrench Nut
  29. 29. © 2014 Megam Systems Watch our screencast #1
  30. 30. © 2014 Megam Systems Data structure Orgs Users Cloud Apps Products prod_id cloud_identity_id users_id org_id api_key accounts nodes predefs Predef clouds /1(org_id) /2(org_id) Cloud Books Histories Cloud Statistics Cloud Books requests Dashboards Widgets
  31. 31. © 2014 Megam Systems Decomposing● Web App – nilavu ● API Server – megam_play ● Cloud Herk – megam_akka ● Gulp – gulp ● Cloud Tool – megam_chef ● Common libs – megam_common ● Ruby API – megam_api ● Cloud Real time – tap ● CLI – meggy Web App CLI API Server Herk Common libs Ruby APICloud Real time Cloud Tool Mob Gulp
  32. 32. © 2014 Megam Systems WebApp ● Ruby on Rails ● Customers will login to https://www.megam.co ● Create an account – Generate a a shared api_key. ● Create Apps for free ● Use default in built cloud settings (EC2). ● Metrics – Meter and Monitor cloud books – Off-line/Real time log ● Dash – Built using AngularJS ● https://github.com/megamsys/nilavu.git ● Status : 0.3 – beta
  33. 33. © 2014 Megam Systems API Server ● Stateless RESTful API Server ● Functional Programming (Scala), Scalaz 7 ● HMAC based Auth. ● Protected resources ● https://api.megam.co ● https://github.com/megamsys/megam_play.git ● Status : 0.1 – beta
  34. 34. © 2014 Megam Systems Gateway
  35. 35. © 2014 Megam Systems Command Line(CLI) ● CLI built using mixlib::cli (ruby), follows the chef cli style. ● Commands – account – nodes (CRUD) – logs ● Status : 0.1 – alpha ● https://github.com/indykish/meggy
  36. 36. © 2014 Megam Systems Cloud Real-Time Streamer ● Real-time streaming server ● Clustered to the number of cores, restarts itself on failure. ● Connects to Redis ● Uses express/socket.io ● Stream content based on URL /streams/:id {where id = a node} ● https://github.com/megamsys/tap.git ● Status : 0.3 beta
  37. 37. © 2014 Megam Systems Redis Tap Monitor Tap Cluster Tap 0 Tap 1 Tap n Redis Tap Cluster Cluster current If fail over n based on core Current If Fail Over Beaver Agent Beaver Agent Beaver Agent Beaver Agent v1/nodes/logs/:id /node0 User clicks on an URL /node1 /noden (or) requests log from pug n1 nn n2
  38. 38. © 2014 Megam Systems Cloud Provisioner ● A facade provisioner which calls a cloud management tool. Supported (Chef11) ● Support for hosted and private chef. ● Instrumented from Herk – Cloud Bridge ● https://github.com/megamsys/megam_chef.git ● Status : 0.3 beta
  39. 39. © 2014 Megam Systems Cloud ProvisionerJSON JSON Validated Async Future Init ~/.megam/megam_chef.yaml { "systemprovider": { "provider": { "prov": "chef" } }, "compute": { "ec2": { "groups": "megam", "image": "ami-56e6a404", "flavor": "m1.small" }, "access": { "ssh-key": "megam_ec2", "identity-file": "~/.ssh/megam_ec2.pem", "ssh-user": "ubuntu" } }, "chefservice": { "chef": { "command": "knife", "plugin": "ec2 server create", "run-list": "'role[opendj]'", "name": "-N TestOverAll" } } } chef_repo tested cookbooks knife-1 knife-2 knife-n . . . . . .
  40. 40. © 2014 Megam Systems Herk – Cloud Bridge ● Main Clustered Akka – Master which manages several slaves of cloud node – pubsub to RabbitMQ ● https://github.com/megamsys/megam_akka.git ● Status : 0.3 beta
  41. 41. © 2014 Megam Systems Gulpd - Native ● Golang based native bridge – Slave that is responsible to mutate a node – pubsub to RabbitMQ ● https://github.com/megamsys/gulp.git ● Status : 0.3 beta
  42. 42. © 2014 Megam Systems Router Seed#1 Zk /machines /instances /nodes /pogo ... vhost 'megam' /megam_conf /pogo(node#1) … /rogo(node#2) Seed#2 Clo#1 Clo#2 Master Worker#1 Master Worker#2 Request Dispatcher#1 Request Dispatcher#3 Request Dispatcher#2 Slave #1 Slave #3Slave #2 CloudProvisioner Gulp #1 Gulp #1
  43. 43. © 2014 Megam Systems How to contribute... Step 1 : Sign our contributor's agreement: Email: gomegam@megam.co.in Step 2: We use Github https://github.com/megamsys Step 3: Setup Dev Environment. Step 4: Contact : gomegam@megam.co.in for our wishlist or better yet suggest one. Step 5. Join our forum https://groups.google.com/forum/?fromgroups#!forum/megamlive

×