CloudStack Hyderabad Meetup: Migrating applications to IaaS clouds


Published on

Keynote by Chirag Jog,
1 November 2012,
CloudStack Hyderabad Meetup,
Lemon Tree, Hyderabad

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

CloudStack Hyderabad Meetup: Migrating applications to IaaS clouds

  1. 1.          InnovaGon  →  ExecuGon  →  SoluGon  →  Delivered     Cloud MigraGng  ApplicaGons  to   IaaS  Clouds   Computing -­‐  Chirag  Jog   Enterprise   DevOps   Applica2ons   Public,  Private,   PaaS,  SaaS   Datacenter   US:            +1  408  556  9645               Management   India:    +91  20  661  43  400             Web:      h8p://     Email:  Clogeny  ConfidenGal  
  2. 2. About  me   Chirag  Jog   •  Co-­‐founder  and  CTO,  Clogeny  Technologies     •  Experience  in  building  some  cuYng  edge  products  in  the  cloud.     •  Extensive  experGse  across  IaaS,  PaaS  &  SaaS  –  20+  cloud  pla]orms   •  Deep  experGse  in  Apache  Cloudstack  Pla]orm   •  Several  deployments  for  marquee  customers    ©  2012  Clogeny  Technologies   h8p://    
  3. 3. Agenda   "  IdenGfying  the  Candidates  for  the  Cloud   "  Choosing  the  right  IaaS  cloud     "  MigraGon  strategy     "  Case  Studies   "  Conclusion  ©  2012  Clogeny  Technologies   h8p://    
  4. 4. IdenGfying  the  Candidates   "  Does  you  applicaGon  have  varying  degree  of  demands?   •  Need  more  horsepower  at  specific  Gmes/seasons   "  Do  you  expect  your  compute,  storage  and  network   requirements  to  grow  over  Gme?   •  Avoid  upfront  costs.   •  Scale-­‐up  when  required.   "  Is  your  tesGng  and  QA  acGvity  restricted  to  the  hardware   you  can  afford?   •  Provision  addiGonal  hardware  only  during  QA/release  cycle   "  Support,  Licensing  costs  ©  2012  Clogeny  Technologies   h8p://    
  5. 5. Criteria  to  choose  the  right  IaaS  Cloud     "  Service  Level  Agreements  (SLAs)   "   ElasGcity  for  Compute  and  Storage  resources   "   Networking  and  Security  features/services   "   Virtual  Machines  and  Data  Backups  and  Restores   "   Availability  Regions  and  Zones   "   DevOps,  AutomaGon   "   Monitoring   "   Image  Management   "   AddiGonal  services  –  Database  as  a  service,  Object   storage,  load  balancers  ©  2012  Clogeny  Technologies   h8p://    
  6. 6. Public  and  Private  Cloud  Providers   "  Leaders   •  Amazon  Web  Services,  Terremark,  Savvis   "  Challengers   •  Bluelock,  Gogrid,  Joyent   " Cloudstack-­‐based   •  Ninefold,  TATA  Instacompute,   "  Private  Clouds   •  Cloudstack,  Eucalyptus,  VMWare  vCloud  Director  and   Openstack  ©  2012  Clogeny  Technologies   h8p://    
  7. 7. MigraGon  Strategy   Investigation  &   Automation  &   Final  Setup   Replication  Phase Optimization   Phase Phase Deployment   Cloud   Cluster  &   Cloud  Asset   Production   Planning Architecture Performance   Setup   Monitoring  &   Analysis   Automation Review   current   and   Arrive   at   the   right   P e r f o r m a n c e   Create   the   components   Standard   and   custom   f u t u r e   b u s i n e s s   topology   that   suits   engineering,   clustering,   and   con>igurations   for   built   reports,   metrics   r e q u i r e m e n t s   a n d   customer   needs,   plan   l o a d   b a l a n c i n g ,   y o u r   c l o u d   a s s e t s   and   parameters   on   develop   relevant   plans   s t o r a g e ,   s e c u r i t y ,   s h a r d i n g ,   c a c h i n g ,   i n c l u d i n g   i m a g e s ,   resource   utilization   scripts,   automation,   for   cloud   adoption   production   automation,   managing  networks  and   and   demand   patterns   m o n i t o r i n g   a n d   including   choice   of   user   monitoring   tools   storage  performance   plus   load   balancer   for   m a n a g e m e n t   t o o l s .   c l o u d   v e n d o r   a n d   for   analysis   of   resource   Con>igure   HA   &   auto-­‐ your   services   on   the   platform usage scaling   cloud©  2012  Clogeny  Technologies   h8p://    
  8. 8. Re-­‐architecGng  for  the  cloud   "  Scalability:  Horizontal,  VerGcal,  Geographical   "   Loose  Coupling   "   Monitoring  and  Logging   "   Automate  and  operaGonalize  as  much  as  possible   "   Design  for  Failure    ©  2012  Clogeny  Technologies   h8p://    
  9. 9. A  case-­‐study:  Social  commerce  pla]orm   The  customer  :  US-­‐based  VC-­‐funded  startup  in  the  social   commerce  and  social  shopping  recommendaGon  space.   Challenges:   "  Hosted  on  non-­‐ElasGc  infrastructure  –  shared  hosGng   "  3rd  party  plugin  in  criGcal  path  –  Highly  Available   "  30-­‐40x  increase  in  traffic  during  holiday  season  –  ability  to   scale-­‐up/down  based  on  load   "  AutomaGon  and  ease-­‐of-­‐use   "  Reduce  TCO  ©  2012  Clogeny  Technologies   h8p://    
  10. 10. A  case-­‐study:  Social  e-­‐commerce  pla]orm   FaceBook     AnalyGcs  Server   Integra;on   Redis    Server     Tornado   WebServer   Redis  Node     Feed  User   Behavior  to   •   Mobile  Apps   Analy;cs   server     • Tablet  Apps   Nginx    FrontEnd   Webserver   Schedule         workers  to   RabbitMQ   Message  Broker     Local  Redis  Cache   perform   • Desktop  Browser     tasks       Pylons  Web     framework       Celery  Worker   MongoDB   User  Interface     StaAc  Content   Javascript/CSS   Serve  Sta;c   Store    all   Content     vendor,   through  a   customer  and   Apache   Webserver   separate   server   10   behavioral   data  in   MongoDB     3rd    Party  Monitoring   (Server  Density)   Exis;ng  Infrastructure        ©  2012  Clogeny  Technologies   h8p://    
  11. 11. MigraGon  Steps  –  InvesGgaGon  Phase   •  Understanding  the  exisGng  architecture.   •  IdenGfy  horizontal  scalable  components.   •  UGlize  Cloudstack’s  features  most  effecGvely.   •  Right-­‐sizing  of  the  servers:   §  Web  servers  :  Mostly  request  forwarders   §  Celery  workers:  CPU  Intensive   §  Datastores  :  Memory  and  CPU  Intensive   •  Storage  configuraGon  for  Volumes:   §  RAID  configuraGon   §  Filesystem  type   §  No.  of  volumes  ©  2012  Clogeny  Technologies   h8p://    
  12. 12. MigraGon  Steps  –  ReplicaGon  Phase   "  Setup  the  servers  and  Volumes  as  per  invesGgaGon   "   Setup  the  webserver  and  workers     "   Database  replicaGon   •  MongoDB  node  set  up  as  slaves  in  replicaGon  mode.   •  Redis  node  are  set  up  as  slaves  in  master-­‐slave  mode.   •  Ensures  all  the  data  –  exisGng  and  live  data  is  replicated   automaGcally.   "  FuncGonal  and  IntegraGon  tesGng  to  ensure  the   applicaGon  works  as-­‐is  ©  2012  Clogeny  Technologies   h8p://    
  13. 13. MigraGon  Steps  –  OpGmizaGon  Phase   "  Database  Layer   •  Convert  MongoDB  from  single-­‐node  to  mulG-­‐node  replica-­‐ set  for  equal  load  distribuGon   •  Built-­‐in  facility  for  fail-­‐over   •  Setup  Redis  as  a  master-­‐slave  with  manual  failover   "  ApplicaGon  Server  Layer   •  Move  all  the  resources  –  message  broker  (RabbitMQ)  and   workers  (Celery)  to  separate  nodes.   •  Ability  to  scale  to  the  workers  (Celery)  based  on  load   "  Web  Server   •  Single  point  of  failure   •  Leverage  Cloudstack’s  Load  Balancer  capabiliGes    ©  2012  Clogeny  Technologies   h8p://    
  14. 14. MigraGon  Steps  –  OpGmizaGon  Phase   "  Monitoring  and  Alerts:   •  Ability  to  monitor  all  the  components     •  track  a  variety  of  metrics  to  pro-­‐acGvely  idenGfy  issues  in   producGon  setup   •  Automated  Email/SMS  alerts  or  Mobile  Apps  for   noGficaGons   •  SoluGons  :  Nagios,  Hyperic,  Cloudkick,  Server  Density   •  Metrics  to  be  tracked  :   §  process  availability   §  server  availability   §  free/used  disk  space   §  free/used  memory   §  free/used  swap  ©  2012  Clogeny  Technologies   h8p://    
  15. 15. MigraGon  Steps  –  AutomaGon  Phase   "  Servers:  Automated  Provisioning,  ConfiguraGon  and   Backup.   "  Data-­‐stores  :  Automated  backups  with  scheduling   "  Self-­‐managing  Virtual  Machines   "  Tools:  Chef,  Puppet,  CfEngine,  Fabric    ©  2012  Clogeny  Technologies   h8p://    
  16. 16. AutomaGon  scripts   "  Provision  all  the  type  of  servers,  configure   appropriate  services  and  storage   "   Setup  a  instance  by  pulling  the  latest  code   "   For  Webservers,  the  tool  configures  the  instance  and   a8aches  it  to  an  Load  Balancer     "   For  MongoDB/Redis:  automaGcally  join  the  cluster  or   create  a  new  one.   "   Data  Backups:   •  Automate  the  backups  of  the  data  stores  arer  quiescing   the  datastore.   •  Runs  arer  a  configurable  period  of  Gme.   •   Manage  exisGng  snapshots  and  delete  really  old  snapshots    ©  2012  Clogeny  Technologies   h8p://    
  17. 17. Cloud-­‐based  Setup     Send  Emails   Regular  backups  of   (for  Alerts)   Volumes  using  snapshots  FaceBook    IntegraGon   Monitoring   AnalyGcs  Server   &  Alerts   Redis    Server   Store  User   Master  –Slave   feeds       Tornado   Mode   WebServer   MongoDB   Feed  User   Replica  Set   Behavior  to   Mode   Analy;cs       server     Redis  Cluster   •   Mobile  Apps       Nginx    FrontEnd     • Tablet  Apps   Webserver   RabbitMQ     Load   Local  Redis  Cache   • Desktop   Balancer   Message     Pylons  Web   Schedule   Broker       framework   workers  to     perform  tasks       MapRed    uce     User  Interface   Celery   StaAc  Content     Worker   Workers     Javascript/CSS   Serve  Sta;c     Load   Content    through  a   17     Balancer   Apache   separate  server   Webserver   Worker  Role   Cloudstack-­‐based  Infrastructure        ©  2012  Clogeny  Technologies   h8p://    
  18. 18. A  case-­‐study:  VMWare  to  Cloudstack   MigraGon   "  The  customer  is  a  leading  managed  soluGon  provider   in  North  America.   "   Challenges:   •  Migrate  100+  VMware  based  VMs  to  Citrix  CloudPla]orm   3.0.5   •  Design  and  develop  a  workflow  to  scan  exisGng  VMWare   vSphere  inventory,  create  images(OVAs)  and  import  into   Cloudstack  as  per  accounts.   •  MulG-­‐volume  OVA  import  into  Cloudstack   •  Restoring  the  associaGon  between  public  IP  address  &   Virtual  Machines  ©  2012  Clogeny  Technologies   h8p://    
  19. 19. A  case-­‐study:  VMWare  to  Cloudstack   MigraGon   "   Key  Features:   •  Developed  a  inventory  scan  engine  using  vSphere  SOAP   APIs    to  generate  a  spec  of  the  complex  hierarchies  of   datacenters,  folders,  hosts  and  clusters.   •  Developed  a  mulG-­‐step  workflow  to:   §   scan  a  virtual  machine  hardware  configuraGon  and  disk  layout   §   generate  a  snapshot  (VMDK)  of  the  virtual  machine   §   extract  per-­‐volumes  OVAs  as  per  Cloudstack’s  expectaGons   from  the  snapshot.   •  Leverage  Cloudstack’s  Lazy  Assignment  of  IP  Addresses  to   allocate  a  specific  IP  address  for  an  account.*     *Don’t  try  this  at  home.  We  modified  the  Cloudstack  database  directly  ©  2012  Clogeny  Technologies   h8p://    
  20. 20. Thanks  !   Contact:  ©  2012  Clogeny  Technologies   h8p://