Direc&ons	
  for	
  CloudStack	
  
Networking	
  
CloudStack	
  	
  SVUG	
  SDN	
  Meetup	
  
September	
  10	
  2015	
  
Chiradeep	
  ViCal	
  
@chiradeep	
  
About	
  me	
  
•  Founding	
  member	
  of	
  cloud.com	
  [	
  ini&al	
  
version	
  of	
  Apache	
  CloudStack]	
  
•  Developed	
  networking	
  and	
  storage	
  
subsystems	
  
•  Developed	
  SDN	
  (GRE	
  overlay),	
  NFV	
  (virtual	
  
router)	
  and	
  group-­‐based	
  policy	
  for	
  CloudStack	
  
•  PMC	
  member	
  of	
  Apache	
  CloudStack	
  
Agenda	
  
•  [Quick]	
  Introduc&on	
  to	
  CloudStack	
  
•  Overview	
  of	
  CloudStack	
  networking	
  
•  CloudStack	
  networking	
  futures	
  
Apache CloudStack is a
•  scalable, 
•  multi-tenant, 
•  open source, 
•  purpose-built,
•  cloud orchestration platform for 
•  delivering turnkey Infrastructure-as-a-
Service clouds
Apache CloudStack
•  Several	
  hundred	
  produc&on	
  clouds	
  
•  Largest	
  clouds	
  in	
  10’s	
  of	
  thousands	
  of	
  
hypervisors	
  
•  Sectors:	
  
• Hos&ng	
  
• Enterprise	
  &	
  Educa&on	
  
• Service	
  Providers	
  
• Web	
  2.0	
  
Commercial	
  and	
  Open	
  Source	
  Success	
  
How can you build your cloud?
Servers
Open Source Xen Hypervisor
Amazon Orchestration Software
AWS API (EC2, S3, …)
Amazon eCommerce Platform
Hypervisor
CloudStack Orchestration Software
Optional Portal
CloudStack or AWS API
StorageNetwork
Networking	
  Concerns	
  
•  Network	
  virtualiza&on	
  	
  
– Mul&-­‐tenancy	
  
•  Network	
  services	
  for	
  virtual	
  networks	
  and	
  
machines	
  
•  Network	
  automa&on	
  
•  Scalability	
  
Networking	
  Principles	
  in	
  Apache	
  
CloudStack	
  
•  Flexibility	
  
–  Allow	
  various	
  combina&ons	
  of	
  technology	
  for	
  L2-­‐L7	
  
network	
  services	
  
–  Allow	
  different	
  providers	
  (vendors)	
  for	
  the	
  same	
  
network	
  service	
  in	
  a	
  Cloud	
  POP	
  
•  Pluggability	
  
–  Plugins	
  allow	
  vendors	
  to	
  drop	
  in	
  vendor-­‐specific	
  
configura&on	
  and	
  lifecycle	
  management	
  code	
  
•  Service	
  scalability	
  
–  Scale	
  out	
  using	
  virtual	
  appliances	
  when	
  possible	
  
–  Scale	
  up	
  using	
  hardware	
  appliances	
  if	
  needed	
  
CloudStack	
  Architecture	
  
Orchestra&on	
  Core	
  
Plugin	
  
Framework	
  
Hypervisor	
  
Plugins	
  
Hypervisor	
  
Plugins	
  
Network	
  Plugins	
  
Network	
  Plugins	
  
Allocator	
  
Plugins	
  
Allocator	
  
Plugins	
  
Storage	
  Plugins	
  
CloudStack	
  Architecture	
  
Orchestra&on	
  Engine	
  
Plugin	
  
Framew
ork	
  
Hyperviso
r	
  Plugins	
  
Hyperviso
r	
  Plugins	
  
Network	
  
Plugins	
  
Network	
  
Plugins	
  
Allocator	
  
Plugins	
  
Storage	
  
Plugins	
  
API	
  
	
   API	
  
	
   API	
  
	
  
Storage	
  
Resource	
  
Physical Resources !
Storage	
  
Resource	
  
Network	
  
Resource	
  Network	
  
Resource	
  
Hypervisor	
  
Resource	
  Hypervisor	
  
Resource	
  
Allocator	
  
Plugins	
  
Allocator	
  
Plugins	
  
1
2
3
4
5
6
7
8
9
Orchestration steps usually executed in sequence!
SDN	
  /	
  Other	
  Overlays/Other	
  Devices	
  
•  Plugins	
  available	
  for	
  
– Midokura	
  
– NVP	
  
– Nuage	
  
– BigSwitch	
  
– Palo	
  Alto	
  
•  GRE	
  /	
  NVGRE	
  on	
  Xen/KVM	
  
•  VxLAN	
  on	
  KVM	
  
Mul&-­‐&er	
  virtual	
  networking	
  
VLAN2724
DB
VM 1!
Web
VM 1!
Web
VM 3!
Web
VM 2!
VLAN101
App
VM 1!
App
VM 2!
VLAN398
!
Virtual Router!
Internet!
Customer!
Premises!
IPSec VPN!
Private Gateway!Loadbalancer	
  
(HW	
  or	
  
Virtual)	
  
Network Services!
•  IPAM!
•  DNS!
•  LB [intra]!
•  S-2-S VPN!
•  Static Routes!
•  ACLs!
•  NAT, PF!
•  FW [ingress & egress]!
Virtual	
  networking	
  with	
  overlays	
  
GREKEY2724
DB
VM 1!
Web
VM 1!
Web
VM 3!
Web
VM 2!
GREKEY101
App
VM 1!
App
VM 2!
GREKEY398
!
VR + vSwitches!
Internet!
Customer!
Premises!
IPSec VPN!
Private Gateway!Loadbalancer	
  
(Virtual)	
  
Network Services!
•  IPAM!
•  DNS!
•  LB [intra]!
•  S-2-S VPN!
•  Static Routes!
•  ACLs!
•  NAT, PF!
•  FW [ingress & egress]!
vSwitch	
  (OVS)	
  used	
  to	
  route	
  
between	
  subnets	
  
Future	
  
•  Containers	
  
•  PaaS	
  
•  SDN	
  solu&ons	
  
•  NFV	
  	
  
•  IPv6	
  
•  Performance	
  
The	
  Narrow	
  Waist	
  Model	
  
of	
  the	
  Internet	
  	
  
Innova&on	
  
Innova&on	
  
Hard	
  to	
  change	
  
Apache	
  CloudStack	
  Narrow	
  Waist	
  
ACS	
  Core	
  
XenServer	
   KVM	
   Hyper-­‐V	
   vSphere	
  
NFS	
  	
   ISCSI	
   FC	
   VLAN	
   Overlay	
   CPU	
  
vCenter	
   libVirt	
   WMI	
   SDN	
  
DbaaS	
  
LBaaS	
  
MRaaS	
  
PaaS	
  
FWaaS	
  
Technology	
  
Applica&ons	
  
Innova&on	
  
Innova&on	
  
Harder	
  to	
  change	
  
Analy&cs*aaS	
   MLaaS	
  
CaaS	
  
Containers	
  on	
  CloudStack	
  
Containers	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  IaaS	
  
•  Containers	
  [run&mes	
  /	
  schedulers	
  /	
  
orchestrators]	
  aim	
  for	
  independence	
  from	
  
underlying	
  infrastructure	
  
– Implement	
  IP	
  address	
  management	
  
– Use	
  overlay	
  networking	
  between	
  containers	
  	
  
– Orchestrate	
  network	
  services	
  such	
  as	
  	
  proxies,	
  
firewalls,	
  port-­‐forwarding	
  
– Volume	
  (persistent	
  logical	
  blobs)	
  orchestra&on	
  
Containers	
  	
  	
  	
  	
  	
  	
  	
  	
  IaaS	
  
•  Containers	
  rely	
  on	
  IaaS	
  for	
  
– Mul&-­‐tenancy	
  
– Network	
  reachability	
  (plumbing)	
  
– Availability	
  of	
  block	
  storage	
  everywhere	
  
– On-­‐demand	
  block	
  storage	
  
– On-­‐demand	
  Container	
  host	
  (VM)	
  scaling	
  
– Network	
  services	
  such	
  as	
  VPN,	
  SSL	
  termina&on	
  
– Failure-­‐domain	
  isola&on	
  
– Affinity	
  /	
  an&-­‐affinity	
  
Containers	
  and	
  IaaS	
  -­‐	
  ques&ons	
  
•  Can	
  containers	
  grow	
  up	
  to	
  be	
  VMs?	
  
– Will	
  container	
  orchestrators	
  replace	
  IaaS	
  ?	
  
•  Can	
  VMs	
  slim	
  down	
  /	
  speed	
  up	
  to	
  have	
  
container-­‐like	
  experiences?	
  
– Will	
  IaaS	
  evolve	
  to	
  address	
  container	
  strengths?	
  
Containers	
  and	
  IaaS	
  -­‐	
  ques&ons	
  
•  Can	
  containers	
  grow	
  up	
  to	
  be	
  VMs?	
  
– Will	
  container	
  orchestrators	
  replace	
  IaaS	
  ?	
  
•  Can	
  VMs	
  slim	
  down	
  /	
  speed	
  up	
  to	
  have	
  
container-­‐like	
  experiences?	
  
– Will	
  IaaS	
  evolve	
  to	
  address	
  container	
  strengths?	
  
•  Can	
  containers	
  and	
  IaaS	
  work	
  together	
  to	
  
reduce	
  inefficiencies?	
  
Overlay	
  on	
  Overlay?	
  
Baremetal	
  to	
  Baremetal	
  
Physical	
  Layer	
  L3	
  Plumbi
VM	
  to	
  VM	
  
Overlay	
  on	
  IP/UDP/TCP	
  
Container	
  to	
  Container	
  
Overlay	
  on	
  Overlay	
  	
  
Docker	
  libNetwork	
  &	
  CloudStack?	
  
•  libnetwork	
  plugins	
  can	
  be	
  used	
  to	
  request	
  
CloudStack	
  network	
  resources:	
  
– IP	
  addresses	
  and	
  MAC	
  addresses	
  
– DNS,	
  DHCP	
  op&ons	
  
•  Requires	
  addi&on	
  to	
  CloudStack	
  APIs.	
  
•  Can	
  poten&ally	
  eliminate	
  overlay-­‐on-­‐overlay	
  
scenarios	
  
Future	
  SDN	
  integra&on	
  
•  OpenDaylight	
  –	
  	
  
–  “modular,	
  extensible,	
  scalable	
  and	
  mul&-­‐protocol	
  
controller	
  infrastructure”.	
  	
  
–  CloudStack	
  Networking	
  plugin	
  can	
  call	
  ODL	
  NB	
  API	
  
•  OVN	
  “opinionated	
  virtual	
  networking”	
  
–  “network	
  virtualiza&on	
  project	
  that	
  brings	
  virtual	
  
networking	
  to	
  Open	
  vSwitch”	
  
–  being	
  developed	
  by	
  the	
  core	
  OVS	
  team.	
  	
  
–  OVN	
  will	
  include	
  logical	
  switches	
  and	
  routers,	
  security	
  
groups,	
  and	
  L2/L3/L4	
  ACLs,	
  implemented	
  on	
  top	
  of	
  a	
  
tunnel-­‐based	
  overlay	
  network	
  
–  CloudStack	
  Networking	
  plugin	
  can	
  call	
  OVN	
  NB	
  API	
  
NFV	
  
•  Apache	
  CloudStack	
  is	
  an	
  early	
  adopter	
  of	
  NFV	
  
to	
  virtualize	
  network	
  services	
  	
  
– DHCP,	
  DNS,	
  L3	
  rou&ng,	
  VPN,	
  LB,	
  FW,	
  etc.	
  
– Knowledge	
  of	
  virtual	
  appliance	
  	
  somewhat	
  
“baked”	
  in	
  however.	
  
•  Ongoing	
  effort	
  to	
  allow	
  other	
  virtual	
  
appliances	
  to	
  integrate.	
  
– Lifecycle	
  management	
  of	
  NFV	
  appliance	
  
– Service	
  chaining	
  of	
  NFV	
  appliances	
  
PaaS	
  
•  PaaS	
  does	
  not	
  require	
  sophis&cated	
  network	
  
services	
  
•  CloudStack’s	
  dual	
  networking	
  models	
  adds	
  to	
  
integra&on	
  challenge	
  
•  CloudFoundry	
  CPI	
  plugin	
  integra&on	
  available	
  
– From	
  NTT	
  (out	
  of	
  date)	
  
– Ongoing	
  work	
  from	
  Orange.	
  
IPv6	
  
•  IPv6	
  addressing	
  available	
  in	
  limited	
  network	
  
configura&ons	
  
•  Work	
  ongoing	
  to	
  add	
  
– IPv6	
  support	
  to	
  Basic	
  Zone	
  (security	
  groups)	
  
– BGP	
  support	
  to	
  exchange	
  routes	
  with	
  external	
  
networks	
  
	
  
Performance	
  
•  Virtual	
  Router	
  performance	
  is	
  always	
  a	
  
wildcard	
  
– Performance	
  varies	
  with	
  infrastructure,	
  
hypervisor,	
  traffic	
  mix	
  
– Hard	
  to	
  op&mize	
  in	
  general	
  
•  Operators	
  would	
  be	
  well	
  served	
  with	
  
guidelines	
  on	
  VR	
  tuning	
  
•  Need	
  a	
  new	
  project	
  for	
  this	
  
Ques&ons?	
  

The Future of SDN in CloudStack by Chiradeep Vittal

  • 1.
    Direc&ons  for  CloudStack   Networking   CloudStack    SVUG  SDN  Meetup   September  10  2015   Chiradeep  ViCal   @chiradeep  
  • 2.
    About  me   • Founding  member  of  cloud.com  [  ini&al   version  of  Apache  CloudStack]   •  Developed  networking  and  storage   subsystems   •  Developed  SDN  (GRE  overlay),  NFV  (virtual   router)  and  group-­‐based  policy  for  CloudStack   •  PMC  member  of  Apache  CloudStack  
  • 3.
    Agenda   •  [Quick]  Introduc&on  to  CloudStack   •  Overview  of  CloudStack  networking   •  CloudStack  networking  futures  
  • 4.
    Apache CloudStack isa •  scalable, •  multi-tenant, •  open source, •  purpose-built, •  cloud orchestration platform for •  delivering turnkey Infrastructure-as-a- Service clouds Apache CloudStack
  • 5.
    •  Several  hundred  produc&on  clouds   •  Largest  clouds  in  10’s  of  thousands  of   hypervisors   •  Sectors:   • Hos&ng   • Enterprise  &  Educa&on   • Service  Providers   • Web  2.0   Commercial  and  Open  Source  Success  
  • 6.
    How can youbuild your cloud? Servers Open Source Xen Hypervisor Amazon Orchestration Software AWS API (EC2, S3, …) Amazon eCommerce Platform Hypervisor CloudStack Orchestration Software Optional Portal CloudStack or AWS API StorageNetwork
  • 7.
    Networking  Concerns   • Network  virtualiza&on     – Mul&-­‐tenancy   •  Network  services  for  virtual  networks  and   machines   •  Network  automa&on   •  Scalability  
  • 8.
    Networking  Principles  in  Apache   CloudStack   •  Flexibility   –  Allow  various  combina&ons  of  technology  for  L2-­‐L7   network  services   –  Allow  different  providers  (vendors)  for  the  same   network  service  in  a  Cloud  POP   •  Pluggability   –  Plugins  allow  vendors  to  drop  in  vendor-­‐specific   configura&on  and  lifecycle  management  code   •  Service  scalability   –  Scale  out  using  virtual  appliances  when  possible   –  Scale  up  using  hardware  appliances  if  needed  
  • 9.
    CloudStack  Architecture   Orchestra&on  Core   Plugin   Framework   Hypervisor   Plugins   Hypervisor   Plugins   Network  Plugins   Network  Plugins   Allocator   Plugins   Allocator   Plugins   Storage  Plugins  
  • 10.
    CloudStack  Architecture   Orchestra&on  Engine   Plugin   Framew ork   Hyperviso r  Plugins   Hyperviso r  Plugins   Network   Plugins   Network   Plugins   Allocator   Plugins   Storage   Plugins   API     API     API     Storage   Resource   Physical Resources ! Storage   Resource   Network   Resource  Network   Resource   Hypervisor   Resource  Hypervisor   Resource   Allocator   Plugins   Allocator   Plugins   1 2 3 4 5 6 7 8 9 Orchestration steps usually executed in sequence!
  • 11.
    SDN  /  Other  Overlays/Other  Devices   •  Plugins  available  for   – Midokura   – NVP   – Nuage   – BigSwitch   – Palo  Alto   •  GRE  /  NVGRE  on  Xen/KVM   •  VxLAN  on  KVM  
  • 12.
    Mul&-­‐&er  virtual  networking   VLAN2724 DB VM 1! Web VM 1! Web VM 3! Web VM 2! VLAN101 App VM 1! App VM 2! VLAN398 ! Virtual Router! Internet! Customer! Premises! IPSec VPN! Private Gateway!Loadbalancer   (HW  or   Virtual)   Network Services! •  IPAM! •  DNS! •  LB [intra]! •  S-2-S VPN! •  Static Routes! •  ACLs! •  NAT, PF! •  FW [ingress & egress]!
  • 13.
    Virtual  networking  with  overlays   GREKEY2724 DB VM 1! Web VM 1! Web VM 3! Web VM 2! GREKEY101 App VM 1! App VM 2! GREKEY398 ! VR + vSwitches! Internet! Customer! Premises! IPSec VPN! Private Gateway!Loadbalancer   (Virtual)   Network Services! •  IPAM! •  DNS! •  LB [intra]! •  S-2-S VPN! •  Static Routes! •  ACLs! •  NAT, PF! •  FW [ingress & egress]! vSwitch  (OVS)  used  to  route   between  subnets  
  • 14.
    Future   •  Containers   •  PaaS   •  SDN  solu&ons   •  NFV     •  IPv6   •  Performance  
  • 15.
    The  Narrow  Waist  Model   of  the  Internet     Innova&on   Innova&on   Hard  to  change  
  • 16.
    Apache  CloudStack  Narrow  Waist   ACS  Core   XenServer   KVM   Hyper-­‐V   vSphere   NFS     ISCSI   FC   VLAN   Overlay   CPU   vCenter   libVirt   WMI   SDN   DbaaS   LBaaS   MRaaS   PaaS   FWaaS   Technology   Applica&ons   Innova&on   Innova&on   Harder  to  change   Analy&cs*aaS   MLaaS   CaaS  
  • 17.
  • 18.
    Containers                    IaaS   •  Containers  [run&mes  /  schedulers  /   orchestrators]  aim  for  independence  from   underlying  infrastructure   – Implement  IP  address  management   – Use  overlay  networking  between  containers     – Orchestrate  network  services  such  as    proxies,   firewalls,  port-­‐forwarding   – Volume  (persistent  logical  blobs)  orchestra&on  
  • 19.
    Containers                  IaaS   •  Containers  rely  on  IaaS  for   – Mul&-­‐tenancy   – Network  reachability  (plumbing)   – Availability  of  block  storage  everywhere   – On-­‐demand  block  storage   – On-­‐demand  Container  host  (VM)  scaling   – Network  services  such  as  VPN,  SSL  termina&on   – Failure-­‐domain  isola&on   – Affinity  /  an&-­‐affinity  
  • 20.
    Containers  and  IaaS  -­‐  ques&ons   •  Can  containers  grow  up  to  be  VMs?   – Will  container  orchestrators  replace  IaaS  ?   •  Can  VMs  slim  down  /  speed  up  to  have   container-­‐like  experiences?   – Will  IaaS  evolve  to  address  container  strengths?  
  • 21.
    Containers  and  IaaS  -­‐  ques&ons   •  Can  containers  grow  up  to  be  VMs?   – Will  container  orchestrators  replace  IaaS  ?   •  Can  VMs  slim  down  /  speed  up  to  have   container-­‐like  experiences?   – Will  IaaS  evolve  to  address  container  strengths?   •  Can  containers  and  IaaS  work  together  to   reduce  inefficiencies?  
  • 22.
    Overlay  on  Overlay?   Baremetal  to  Baremetal   Physical  Layer  L3  Plumbi VM  to  VM   Overlay  on  IP/UDP/TCP   Container  to  Container   Overlay  on  Overlay    
  • 23.
    Docker  libNetwork  &  CloudStack?   •  libnetwork  plugins  can  be  used  to  request   CloudStack  network  resources:   – IP  addresses  and  MAC  addresses   – DNS,  DHCP  op&ons   •  Requires  addi&on  to  CloudStack  APIs.   •  Can  poten&ally  eliminate  overlay-­‐on-­‐overlay   scenarios  
  • 24.
    Future  SDN  integra&on   •  OpenDaylight  –     –  “modular,  extensible,  scalable  and  mul&-­‐protocol   controller  infrastructure”.     –  CloudStack  Networking  plugin  can  call  ODL  NB  API   •  OVN  “opinionated  virtual  networking”   –  “network  virtualiza&on  project  that  brings  virtual   networking  to  Open  vSwitch”   –  being  developed  by  the  core  OVS  team.     –  OVN  will  include  logical  switches  and  routers,  security   groups,  and  L2/L3/L4  ACLs,  implemented  on  top  of  a   tunnel-­‐based  overlay  network   –  CloudStack  Networking  plugin  can  call  OVN  NB  API  
  • 25.
    NFV   •  Apache  CloudStack  is  an  early  adopter  of  NFV   to  virtualize  network  services     – DHCP,  DNS,  L3  rou&ng,  VPN,  LB,  FW,  etc.   – Knowledge  of  virtual  appliance    somewhat   “baked”  in  however.   •  Ongoing  effort  to  allow  other  virtual   appliances  to  integrate.   – Lifecycle  management  of  NFV  appliance   – Service  chaining  of  NFV  appliances  
  • 26.
    PaaS   •  PaaS  does  not  require  sophis&cated  network   services   •  CloudStack’s  dual  networking  models  adds  to   integra&on  challenge   •  CloudFoundry  CPI  plugin  integra&on  available   – From  NTT  (out  of  date)   – Ongoing  work  from  Orange.  
  • 27.
    IPv6   •  IPv6  addressing  available  in  limited  network   configura&ons   •  Work  ongoing  to  add   – IPv6  support  to  Basic  Zone  (security  groups)   – BGP  support  to  exchange  routes  with  external   networks    
  • 28.
    Performance   •  Virtual  Router  performance  is  always  a   wildcard   – Performance  varies  with  infrastructure,   hypervisor,  traffic  mix   – Hard  to  op&mize  in  general   •  Operators  would  be  well  served  with   guidelines  on  VR  tuning   •  Need  a  new  project  for  this  
  • 29.