Your SlideShare is downloading. ×
CloudStack Architecture and Refactor
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

CloudStack Architecture and Refactor

839
views

Published on

Describe the latest CloudStack architecture and the undergoing refector.

Describe the latest CloudStack architecture and the undergoing refector.

Published in: Design, Technology, Business

1 Comment
0 Likes
Statistics
Notes
  • thx to share, but page9,can't see the words
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total Views
839
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
1
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. CloudStack  Architecture  and  its  evolu4on  –  A  developer’s  view  5/25/2013  Kelven  Yang  Citrix    
  • 2. Outline  •  Overview  of  CloudStack    •  General  architecture  abstrac4on  •  CloudStack  Cloud  architecture  •  CloudStack  Cloud  subsystems  •  CloudStack  Provision-­‐automa4on  ingredients  •  CloudStack  System  VMs  •  Architecture  refactoring  •  Q/A    
  • 3. CloudStack  Overview  
  • 4. •  Secure,  mul4-­‐tenant  cloud  orchestra4on  plaOorm  –  Turnkey  plaOorm  for  delivering  IaaS  clouds  –  Hypervisor  agnos4c  –  Scalable,  secure  and  open  –  Open  source,  open  standards  –  Deploys  on  premise  or  as  a  hosted  solu4on  •  Deliver  cloud  services  faster  and  cheaper  What  is  CloudStack?  Build  your  cloud  the  way  the  world’s  most  successful  clouds  are  built  
  • 5.    Compute  CloudStack  Provides  On-­‐demand  Access  to  Infrastructure  Through  a  Self-­‐Service  Portal          Network      Storage  Admin Users Org A Admin Users Org B Users End User Admin
  • 6. Open  Flexible  PlaOorm      Compute      XenServer   VMware   KVM  Oracle  VM   Bare  metal  Hypervisor      Storage      Local  Disk   iSCSI   NFS  Fiber  Channel  SwiY  Block  &  Object      Network      Network  Type  Isola4on  Load  balancer  Firewall   VPN  Network  &  Network  Services  Primary  Storage   Secondary  Storage  
  • 7. General  Architecture  Abstrac4on  
  • 8. General  Architecture  Abstrac4on  •  Resource  Agent  Endpoint  for  CloudStack  to  communicate  with  underlying  virtual  or  physical  resource,  example  of  resource  could  be  a  hypervisor  host  or  a  network  element    •  CloudStack  RPC  (message  bus)  Cross  component  interac4on  Cross  management  server  cluster    •  Asynchronous  job  execu4on  Orchestra4on  work-­‐flow  usually  takes  4me  to  finish,  not  friendly  to  synchronous  API  paradigm      
  • 9. General  Architecture  Abstrac4on  Business  Logic  Modules  Business  Logic  Module  Business  Logic  Module  API  entry    Job  control  ORM  Database  Object  Layer  Agent  Agent  Agent  Agent  
  • 10. General  Architecture  Abstrac4on  Business  Logic  Module  Direct  Agent  Co-­‐located  Agent  Direct  agent  –  agent  running  inside  management  server  Co-­‐located  agent  –  agent  that  is  co-­‐located  with  external  resource  (i.e.  Host)  KVM  XenServer  Host  XAPI  
  • 11. CloudStack  Cloud  Architecture  
  • 12. •  Hosts  •  Servers  onto  which  services  will  be  provisioned  •  Primary  Storage  •  VM  storage  •  Cluster  •  A  grouping  of  hosts  and  their  associated  storage  •  Pod  •  Collec4on  of  clusters  •  Network  •  Logical  network  associated  with  service  offerings  •  Secondary  Storage  •  Template,  snapshot  and  ISO  storage  •  Zone  •  Collec4on  of  pods,  network  offerings  and  secondary  storage  •  Management  Server  Farm  •  Responsible  for  all  management  and  provisioning  tasks  Core  CloudStack  Components  Zone  CloudStack  Pod  Cluster    Host    Host  Network  Primary  Storage  VM  VM  CloudStack  Pod  Cluster  Secondary  Storage  
  • 13.                                                                                                            Pod  1  ….  Cluster  N  Access LayerHost 2Cluster  1  Conceptual  Deployment  Architecture  Host 1Ø  Hypervisor  is  the  basic  unit  of  scale.    Ø  Cluster  consists  of  one  ore  more  hosts  of  same  hypervisor    Ø  All  hosts  in  cluster  have  access  to  shared  (primary)  storage  Ø  Pod  is  one  or  more  clusters,  usually  with    L2  switches.    Ø  Availability  Zone  has  one  or  more  pods,  has  access  to  secondary  storage.    Ø  One  or  more  zones  represent  cloud    Primary  Storage  Zone  1  ….  L3 core  Secondary  Storage  Pod  N  CloudStack  Management  Server  Internet  
  • 14. Management  Server  Deployment  Architecture    -­‐  Do  Not  Distribute  Management  Server  MySQL  DB  Back  Up  DB  Infrastructure  Resources  User  API  Admin  API  Load  Balancer  Management  Server  Management  Server  Management  Server  MySQL  DB  Infrastructure  Resources  User  API  Admin  API  Single-­‐node  Deployment  Mul4-­‐node  Deployment  Ø  MS  is  stateless.  MS  can  be  deployed  as  physical  server  or  VM  Ø  Single  MS  node  can  manage  up  to  10K  hosts.  Mul4ple  nodes  can  be  deployed  for  scale  or  redundancy  Ø  Commercial:  RHEL  5.4+;  FOSS:  Ubuntu  10.0.4,  Fedora  16  Replica4on  
  • 15. CloudStack  Cloud  Subsystems  
  • 16. Compute  Subsystem  VirtualMachineGuru  HypervisorGuru  ServerResource  UserVm  Guru  System  VM  Guru  System  VM  Gurus  Hypervisor  Gurus  Hypervisor  Gurus  XenServer  Resource  Vmware  Resource  KVM  Resource  Server  context  Agent  context  Orchestra4on  flow  RPC  (Message  Bus)  
  • 17. Compute  Subsystem  •  VirtualMachineGuru  Defines  pluggable  points  for  various  VM  managers  to  implement  specific  VM  orchestra4on  logic.    UserManagerManager,  ConsoleProxyManager  etc  implements  this    Interface.    •  HypervisorGuru  Defines  pluggable  points  to  insert  hypervisor  specific  logic  into  the  Overall  orchestra4on  flow.    •  ServerResource  Defined  the  abstrac4on  layer  for  various  hypervisor  resource  agent  to    realize  the  ac4on  needed  from  orchestra4on  flow      
  • 18. Storage  Subsystem  •  Subsystem  func4ons  §  Provision  VM  volume  to  storage  devices  §  Manage  volume  snapshots  §  Manage  Templates  and  ISOs  §  Mo4on  service  to  move  storage  content    •  Storage  classifica4on  §  Primary  Storage  Provides  VM  volume  storage  at  all  4me  including  run4me  Demand  high  performance  storage  I/O  for  read/write  access  §  Secondary  Storage  (Backup/Object  store)  Store  ready-­‐only  content  (templates,  ISOs)      
  • 19. •  Primary  Storage  •  Cluster  level  storage  for  VMs  •  Connected  directly  to  hosts  •  NFS,  iSCSI,  FC  and  Local  •  Secondary  Storage  •  Zone  level  storage  for  template,  ISOs  and  snapshots  •  NFS  or  OpenStack  SwiY  via    CloudStack  System  VM  •  Templates  and  ISOs  •  Imported  into  CloudStack  •  Can  be  private  or  public  Understanding  roles  in  Storage    Zone  Secondary  Storage  Pod  Cluster    Host    Host  Primary  Storage  Template  
  • 20. Network  subsystem  •  Subsystem  func4ons  §  Networking  mul4-­‐tenancy  §  Provision  user  logic  network  §  Manage  physical/logical  network  configura4on  §  Provide  pluggable  framework  for  third-­‐party  vendors  
  • 21. CloudStack  Management  Network  Management  Server  System  VM  User  VM  172.16.0.10  172.16.0.15  172.16.0.20  Hypervisor  Host  Management  server  manages  physical  or  virtual  resource  through  management  network(s).    The  network  that  is  used  to  manage  hypervisor  host  can  be  different  to  the  network  that  is  used  to  management  system  VMs    
  • 22. Host  link-­‐local  network  Management  Server  System  VM  User  VM  172.16.0.10  172.16.0.15  Hypervisor  Host  169.254.0.1  169.254.0.10  Why  host  link  local  network?  •  For  hypervisor  hosts  that  have  co-­‐located  agent,  host  link-­‐local  network  can  reduce  IP  address  usage  •  Hypervisor  co-­‐located  agent/plugin  proxy  the  channel  between  virtual  resource  (system  VM)  and  management  server  
  • 23. Mul4-­‐tenancy:  L2  VLAN  isola4on  CloudStack  Management  Server  System  VM  Host  Dom0/Kernel  Guest  VM  Hypervisor  Host  System  VM  Host  Dom0/Kernel  Guest  VM  Hypervisor  Host  CloudStack  Management  Server  192.168.10.0/24  10.1.0.0/24  on  VLAN  100  10.1.0.0/24  on  VLAN  200  
  • 24. Mul4-­‐tenancy:  Security  group  isola4on  …  DB    Security  Group  Web  Security  Group  …   …  Web  VM  Web  VM  Web  VM  Web  VM  DB  VM  Web  VM  DB  VM  Web  VM  
  • 25. Network  Provisioning  NeworkGuru  GuestNetworkGuru  NetworkElement  VirtualRouterElement  NetScalarElement  RPC(Message  Bus)  Network  orchestra4on  flow  Network  Offering  DirectNetworkGuru  PhysicalNetwork  Configura4on  NetworkGuru    -­‐  helps  define,  implement  shared  or  a  tenant  guest  network  in  a  lazy  manner,    it  also  helps      perform  resource  (IP,  MAC  address,  SDN  tenant  ID)  management  in  various  orchestra4on      phases  NetworkElement  –  carries  on  network  related  ac4ons    PhysicalNetwork  –  defines  a  mapping  configura4on  to  help  map  logic  network  into  physical  infrastructure  Network  Offering  –  define  a  feature  set  template  for  guest  network,  help  instan4ate  network  instance  in  lazy      construc4on  stage    
  • 26. CloudStack  Provision-­‐automa4on  ingredients  
  • 27. Offerings  •  Service  Offering  CPU  speed/CPU  core/Memory/HA/Rate  Limit/Tags  •  Disk  Offering  Disk  size/Local  Storage/Tags  •  Network  Offering  Rate  limit/Traffic  Type/Isola4on  characteris4cs/feature  masks    All  types  of  offerings  enable  CloudStack  to  instan4ated  related  object  in  a  lazy,    “fluid”  manner  
  • 28. Tags  •  Tagging  resource  objects(Host,  Storage,  etc)  •  Tagging  offering  objects(ServiceOffering,  DiskOffering,  etc)  Perform  loosely-­‐coupled  associa4on  between  objects,  typically  between  offering  Objects  and  resource  objects,  resource  allocators  can  take  advantage  of  the    associa4on  to  influent  alloca4on  affinity    
  • 29. CloudStack  System  VMs  
  • 30. CloudStack  System  VMs  •  System  VMs  op4mize  and  scale  the  data-­‐path  on  behalf  of  CloudStack  –  Stateless,  can  be  destroyed  and  recreated  from  database  state  –  Highly  Available  –  Communicates  with  Management  Server  over  management  network  –  Usually  have  3  interfaces:  control,  guest  and  public  •  Console  Proxy  VM    –  Provides  AJAX-­‐style  HTTP-­‐only  console  viewer  –  Grabs  VNC  output  from  hypervisor  –  Scales  out  (more  spawned)  as  load  increases  –  Java-­‐based  server  Communicates  with  MS  over  message  bus  •  Secondary  Storage  VM  –  Provides  image  (template)  management  services  –  Download  from  HTTP  file  share  or  SwiY  –  Copy  between  zones  –  Scale  out  to  handle  mul4ple  NFS  mounts  –  Java-­‐based  server  communicates  with  MS  over  message  bus  
  • 31. System  VM  contd  •  SSH  keys  and  password  are  unique  to  cloud  installa4on  •  Code  can  be  patched  by  restar4ng  system  vm  –  Mounts  a  special  ISO  file  with  latest  code  at  boot  –  If  ISO  contents  differ,  patch  and  reboot  •  Same  system  vm  works  on  XS,  KVM,  VMWare  –  Bootstrap  step  for  the  cloud  is  to  install  the  template  for  this  system  vm  •  Ready  to  be  re-­‐purposed  for  other  specialized  tasks    
  • 32. Architecture  refactoring  
  • 33. Architecture  refactoring  •  Good  about  CloudStack  Ø   Simplicity  Easy  to  understand    Easy  to  setup  Easy  to  operate  Out  of  box  solu4on  to  almost  any  Cloud,  from  private  to  public  cloud  •  Bad  about  CloudStack  Ø Tightly-­‐coupled  This  architecture  complain  is  being  addressed  in  Apache  CloudStack  community  
  • 34. Architecture  refactoring  •  Modular/Componen4za4on  refactoring  –  Build  system  to  maven  –  Adopt  Spring  Framework  •  Make  component  wiring  consistent  and  explicit  •  Loosely-­‐coupling  –  RPC/Message  Bus  improvement  •  Async  programming  model  –  AsyncCallFuture<T>  •  VM  state  sync  
  • 35. Architecture  refactoring  •  RPC/Message  Bus  improvement  –  Interface  binding  AgentManager/Listener    AgentManager  defines  the  RPC/Messaging  API,  users    of  RPC/  Messaging  implement  Listener  interface  and    expliclitly  bind  to    AgentManager  –  Topic  binding  Topic  constants    Users  subscribe  to  topic  itself,  publisher  and  subscriber  are        associated  indirectly  through  topic  constants    –  Hierarchical  topic  naming  conven4on  Topic  to  be  similar  as  DNS  names,  subscribers  at  higher  level  can  be  no4fied  for  down-­‐level  events  –  Transparently  extend  message  bus  to  cross  boundaries    
  • 36. Architecture  refactoring  •  Async  programming  model      public  void  installCallback()  {                    TestContext<String>  context  =  new  TestContext<String>(null);                    AsyncCallbackDispatcher<AsyncSampleEventDrivenStyleCaller,  Object>  caller  =            AsyncCallbackDispatcher.create(this);                    caller.setCallback(caller.getTarget().HandleVolumeCreateAsyncCallback(null,null)).setContext(context);                      String  test  =  "test";                        _ds.createVolumeAsync(test,  caller);                        Assert.assertEquals(test,  context.getResult());            }            protected  Void    HandleVolumeCreateAsyncCallback(AsyncCallbackDispatcher<AsyncSampleEventDrivenStyleCaller,        String>  callback,  TestContext<String>  context)  {                      String  resultVol  =  callback.getResult();                      context.setResult(resultVol);                      return  null;              }    
  • 37. Architecture  refactoring  •  AsyncCallFuture<T>  to  connect  sync  and  async  worlds    public  void  MethodThatWillCallAsyncMethod()  {              String  vol  =  new  String("Hello");            AsyncCallbackDispatcher<AsyncSampleEventDrivenStyleCaller,  Object>  caller  =  AsyncCallbackDispatcher.create(this);                    AsyncCallFuture<String>  future  =  _ds.createVolume(vol);                    try  {                            String  result  =  future.get();                            Assert.assertEquals(result,  vol);                    }  catch  (InterruptedExcepOon  e)  {                            //  TODO  Auto-­‐generated  catch  block                            e.printStackTrace();                    }  catch  (ExecuOonExcepOon  e)  {                            //  TODO  Auto-­‐generated  catch  block                            e.printStackTrace();                    }            }  
  • 38. Architecture  Refactoring  Current  VMSync  implementa?on  •  Basic  sync  unit  at  host/cluster  level  •  ini4alize  sync-­‐scope  at  startup  (full  sync)  •  monitor  and  smoothen  state  changes  and  perform  delta  reports  –  Resource  needs  to  know  about  CloudStack  specific  VM  state  like  migra4ng,  star4ng  –  Resource  needs  to  smoothen  state  report  in  reboo4ng  case,  if  reboot  is  issued  from  guest  OS  •  Threads  from  mul4ple  sources  may  collide  when  performing  state  changes  –  Thread  from  HA/State  recovery  procedure  –  Thread  from  API  request  –  Thread  from  Host  VM  state  report  
  • 39. Architecture  Refactoring  Basic  idea  of  VMSync  refactoring  •  Resource  agent  is  only  required  to  report  VM  raw  power  state  •  CloudStack  VM  state  transi4on  is  driven  by  state-­‐transi4on  jobs,  avoid  mul4ple  non-­‐coorporated  driving  sources  (from  HA/API/sync  report)  •  Opera4on  job  on  a  VM  is  serialized  for  execu4on  •  Conflict  resolu4on  –  Automa4c  policy  (HA/force  sync)  –  Manual  policy  (Alerts,  Admin  interven4on,  user  acknowledgement  workflow)  
  • 40. Thank  you  •  Q/A  •  For  more  informa4on,  please  visit  h}p://cloudstack.apache.org/    

×