Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Challenges	
  and	
  lessons	
  learned	
  with	
  
Openstack	
  deployments	
  and	
  MySQL	
  
Sandro	
  Mazzio+a	
  
Sr...
Who	
  I	
  am	
  
	
  
o  I	
  am	
  not	
  a	
  DB	
  expert	
  or	
  a	
  DBA	
  
o  I	
  work	
  in	
  Product	
  Mana...
San Francisco
Montréal
Paris
Bangalore
eNovance	
  loca>ons	
  and	
  figures	
  
People
	
  
	
  
	
  
Customers	
  
	
  
...
Value	
   Engineering	
  
Software
Software Architecture : from centralized apps to distributed services
 Software Enginee...
Market	
  &	
  opportunity	
  
	
  
	
  Large
Enterprises
Software for Private Cloud:
$ 10 bn by 2017

Telcos
xSP
Software...
Build a Cloud
Deliver your Private or Public
Cloud on Premise with a real
Cloud Approach :
•  No Versioning
•  Pay as you ...
Openstack	
  &	
  eNovance	
  
	
  
	
  
	
  
q Gold	
  Member	
  
q 	
  Top	
  Ten	
  Contributor	
  
q Share	
  DNA	
...
Openstack	
  is	
  not	
  simple	
  
	
  
Openstack	
  is	
  not	
  cloud	
  	
  
(Subbu	
  Allamaraju	
  eBay,	
  July	
  25,	
  2013)	
  
All	
  key	
  requiremen...
Openstack	
  Upstream	
  does	
  not	
  address	
  HA	
  
	
  
Openstack	
  HA	
  Guide	
  
	
  
h+p://docs.openstack.org/...
eNovance	
  Openstack	
  Reference	
  Architecture	
  V1.0	
  
	
  
Ø Management	
  cluster	
  full	
  highly	
  availabl...
eNovance	
  Openstack	
  product	
  
	
  
•  Contains	
  all	
  core	
  projects	
  of	
  
Openstack	
  
•  Supports	
  HA...
What	
  about	
  DB	
  in	
  our	
  product?	
  
DB	
  Requirements:	
  
FuncRonal	
  &	
  Performance	
  requirements:	
 ...
eNovance	
  Galera	
  for	
  MySQL	
  usage:	
  results	
  
	
  
Experience	
  from	
  ProducRon	
  Environment:	
  
1	
  ...
On	
  the	
  topic	
  of	
  Mul>	
  Data	
  Center	
  
	
  
West 1
East
EMEA
ASIA
Goal	
  is	
  to	
  address:	
  
	
  
• ...
Not	
  all	
  services	
  are	
  ready	
  for	
  Mul>-­‐DC	
  
	
  
•  Openstack	
  has	
  a	
  concept	
  of	
  AZ,	
  Ho...
Keystone	
  
	
  
Our  experience  working  at  large  
customers  is  that:

•  Token  storage  is  a  nightmare  and
•  ...
Everything	
  is	
  MySQL,	
  almost…	
  
	
  
VS	
  
Ceilometer,	
  we	
  use	
  mongoDB	
  
	
  
But	
  
Requires	
  MongoDB	
  DBA,	
  Runbook,	
  skills,	
  
…	
  
Because	...
Openstack	
  is	
  not	
  cloud	
  	
  
(Subbu	
  Allamaraju	
  eBay,	
  July	
  25,	
  2013)	
  
All	
  key	
  requiremen...
Why	
  we	
  need	
  update	
  –	
  upgrade	
  –patching…	
  ?	
  
	
  
User	
  
CRM	
  
Billing	
  
Portal	
  
seongs	
  ...
What	
  is	
  our	
  vision	
  
	
  
Deliver	
  Openstack	
  and	
  app	
  on	
  top	
  of	
  
it	
  using	
  Openstack	
 ...
Leverage	
  the	
  Openstack	
  process	
  
	
  
How	
  Openstack	
  handle	
  
•  Contribu/on  of  400  developers  /  mo...
…	
  and	
  tools	
  
	
  
Everything  in
•  Process  descrip/on  and  code
•  Infrastructure  used  to  handle  the  proc...
Benefits	
  	
  
	
  
Improve	
  Efficiency	
  
	
  
•  Nothing  is  local,  everything  is  shared
•  Control  access  right...
What	
  we	
  are	
  describing	
  is	
  Devops…	
  
	
  
•  Validate	
  code	
  changes	
  and	
  mulRple	
  deployments	...
Conclusion	
  
	
  
•  Openstack	
  is	
  not	
  easy.	
  
•  It	
  is	
  ok	
  at	
  small	
  scale	
  in	
  a	
  single	...
Ques>ons?	
  

	
  
	
  
	
  
Sandro  Mazzioaa
sandro.mazzioaa@enovance.com
@smazzioaa
Upcoming SlideShare
Loading in …5
×

Percona presentation v2

409 views

Published on

Published in: Technology, Business
  • Be the first to comment

Percona presentation v2

  1. 1. Challenges  and  lessons  learned  with   Openstack  deployments  and  MySQL   Sandro  Mazzio+a   Sr  Director  Product  Management  IaaS  
  2. 2. Who  I  am     o  I  am  not  a  DB  expert  or  a  DBA   o  I  work  in  Product  Management  since  15+  years   o  I  work  at  eNovance  since  2  months   o  I  started  with  Openstack  in  July  2011   o  I’ve  been  launching  HP  Cloud  Services  DBaaS  offering  
  3. 3. San Francisco Montréal Paris Bangalore eNovance  loca>ons  and  figures   People       Customers        Growth in Revenue   Founded in      
  4. 4. Value   Engineering   Software Software Architecture : from centralized apps to distributed services Software Engineering : from Waterfall to Agile Infrastructure IT Infrastructure : from Mainframe to Cloud Cloud Infrastructure Engineering : from Ops to DevOps Journey  to  the  cloud     Client / Server Vertical Scaling Hardware is key Centralized Services / API Horizontal Scaling Software is key Distributed Expensive Hw Vertical Scaling Monolithic Centralized Cheap Hw Horizontal Scaling Distributed
  5. 5. Market  &  opportunity      Large Enterprises Software for Private Cloud: $ 10 bn by 2017 Telcos xSP Software for Public Cloud : $ 8 bn by 2017 Telecom Equipment Vendors NFV : $ 2bn by 2017 Not Scalable Expensive Not Cloud Rich Set of Features No Versioning Shared Infrastructure Not Scalable Expensive Not Cloud Containers Not Scalable Not Cloud Migrate from Physical Appliance To Network Functions Virtualization eNovance   Cloud  Solu/ons Seamless  Build  &  Delivery   of  OpenStack  Cloud Infrastructures
  6. 6. Build a Cloud Deliver your Private or Public Cloud on Premise with a real Cloud Approach : •  No Versioning •  Pay as you Grow •  AWS Features Match •  Enterprise Grade SLA •  24/7 Support Use a Cloud   Deliver Cloud-Ready applications on top of existing Cloud infrastructures : •  Elastic & Scalable •  Continuous Delivery •  Pay as you Grow •  Enterprise Grade SLA •  24/7 Support     What  we  do  ?        
  7. 7. Openstack  &  eNovance         q Gold  Member   q   Top  Ten  Contributor   q Share  DNA  Open  Cloud,  Open  Source  
  8. 8. Openstack  is  not  simple    
  9. 9. Openstack  is  not  cloud     (Subbu  Allamaraju  eBay,  July  25,  2013)   All  key  requirements  for  producRon:   •  By  Enterprise  for  Private  Cloud   •  By  Service  Providers  for  Public  Cloud   “infra  onboarding,  boostrapping,  remediaRon,  config  management,  patching,  packaging,  upgrades,  high  availability,   monitoring,  metrics,  user  support,  capacity  forecasRng  and  management,  billing  or  chargeback,  reclamaRon,  security,   firewalls,  DNS,  integraRon  with  other  internal  infrastructure  and  tools,  and  on  and  on  and  on”.  
  10. 10. Openstack  Upstream  does  not  address  HA     Openstack  HA  Guide     h+p://docs.openstack.org/high-­‐ availability-­‐guide/content/ index.html   ApplicaRon  on  top  Openstack   OS  Infra.  Plane:    Openstack  API    Message  Queues    DB   VirtualizaRon  Layer   …  
  11. 11. eNovance  Openstack  Reference  Architecture  V1.0     Ø Management  cluster  full  highly  available   Ø  OpenStack  API,  schedulers   Ø  RabbitMQ,  Database,  Cache   Ø  Network  services   Ø Compute  nodes  with  KVM  and  Open-­‐ vSwitch  with  GRE  tunnels  as  overlay   Ø Swia  cluster   Ø Ceph  cluster,  used  as  backend  for  Nova,   Glance  and  Cinder  
  12. 12. eNovance  Openstack  product     •  Contains  all  core  projects  of   Openstack   •  Supports  HA  in  a  Single  DC  mode   •  Based  on     •  eDeploy  (eNovance  Bare  Metal   Deployment  Opensource)   •  Puppet  as  Soaware   ConfiguraRon  Management  
  13. 13. What  about  DB  in  our  product?   DB  Requirements:   FuncRonal  &  Performance  requirements:     •  100  %  upRme   •  Performance  DegradaRon  is  ok     Business  Requirements:   Opensource   Galera  cluster  for  MySQL  
  14. 14. eNovance  Galera  for  MySQL  usage:  results     Experience  from  ProducRon  Environment:   1  cluster  of  3  Management  Nodes   •  No  Service  InterrupRon   •  Performance  DegradaRon  are  OK   •  Galera  performed  correctly  
  15. 15. On  the  topic  of  Mul>  Data  Center     West 1 East EMEA ASIA Goal  is  to  address:     •  Disaster  Recovery   •  Local  Affinity  
  16. 16. Not  all  services  are  ready  for  Mul>-­‐DC     •  Openstack  has  a  concept  of  AZ,  Host  AggregaRon  =  good  in  a  single  DC  .   Not  good  If  your  DC  are  separated  by  a  WAN…   •  Nova  Cells  is  good  for  distribuRon  (but  nobody  else  is  taken  care)…   •  MulRple  Region  –  Keystone  in  every  region  or  one  overlay     Galera  3   Galera  SynchronizaRon  over   WAN   Using  Heat  to  orchestrate   MulRple  Region  
  17. 17. Keystone     Our  experience  working  at  large   customers  is  that: •  Token  storage  is  a  nightmare  and •  and  do  not  scale  well  on  large  scale   deployment   •  Key  Value  Store  (memcache,  redis,   …)  -­‐>  MySQL  -­‐>  memcache  (but  we   have  a  bug)…  euh…
  18. 18. Everything  is  MySQL,  almost…     VS  
  19. 19. Ceilometer,  we  use  mongoDB     But   Requires  MongoDB  DBA,  Runbook,  skills,   …   Because  we  need  scalibility  and  because   of  the  nature  of  informaRon  to  store  
  20. 20. Openstack  is  not  cloud     (Subbu  Allamaraju  eBay,  July  25,  2013)   All  key  requirements  for  producRon:   •  By  Enterprise  for  Private  Cloud   •  By  Service  Providers  for  Public  Cloud   “infra  onboarding,  boostrapping,  remediaRon,  config  management,  patching,  packaging,  upgrades,  high  availability,   monitoring,  metrics,  user  support,  capacity  forecasRng  and  management,  billing  or  chargeback,  reclamaRon,  security,   firewalls,  DNS,  integraRon  with  other  internal  infrastructure  and  tools,  and  on  and  on  and  on”.  
  21. 21. Why  we  need  update  –  upgrade  –patching…  ?     User   CRM   Billing   Portal   seongs   changes   new   release   bug  fixes   driver  firmware   update   IntegraRon   mess  
  22. 22. What  is  our  vision     Deliver  Openstack  and  app  on  top  of   it  using  Openstack  process  and  tools   ConRnuous  IntegraRon/Delivery  
  23. 23. Leverage  the  Openstack  process     How  Openstack  handle   •  Contribu/on  of  400  developers  /  month? •  Developers  from  250  companies? •  10+  integrated  projects? and  yet  release  a  new  version   •  every  6  months   •  with  400  new  features   •  without  major  inconsistencies?  
  24. 24. …  and  tools     Everything  in •  Process  descrip/on  and  code •  Infrastructure  used  to  handle  the  process •  Code  to  deploy  the  infrastructure 100  Rmes  be+er  than  documentaRon  of  a  manual   process  that  is  never  up  to  date...   Reproducible  environments     •  Dynamic  Provision  test  env  when   needed  by  Jenkins •  Allow  dev  to  generate  a  test  env  for   debug  at  will •  Store  validated  releases  and  customer   specific  env  (in  Openstack  Swi^)
  25. 25. Benefits       Improve  Efficiency     •  Nothing  is  local,  everything  is  shared •  Control  access  rights •  Contribute  model •  No  need  for  a  full  release  team •  Factorize  hardware  needs Reproducible  Env     •  Dynamically  provision  test   environments  when  needed  by   Jenkins •  Allow  developers  to  generate  a  test   environment  for  debugging  at  will •  Store  validated  releases  and   customer  specific  environments  (in   Swi^)
  26. 26. What  we  are  describing  is  Devops…     •  Validate  code  changes  and  mulRple  deployments  environments  at  once   •  Accommodate  more  devs  working  on  the  same  projects   •  Limit  wait  Rmes     ⇒  Industrializes  development  and...  
  27. 27. Conclusion     •  Openstack  is  not  easy.   •  It  is  ok  at  small  scale  in  a  single  DC  mode.     •  eNovance     •  has  a  product  to  address  Single  DC  HA  mode  deployment   •  is  applying  Openstack  CD/CI  to  Openstack  deployment  itself   •  is  working  on  the  MulR  DC  mode  
  28. 28. Ques>ons?         Sandro  Mazzioaa sandro.mazzioaa@enovance.com @smazzioaa

×