Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Tutorial	
  on	
  Contribu-ng	
  to	
  
OpenStack	
  
Devda8a	
  Kulkarni	
  
PTL,	
  OpenStack	
  Solum	
  
Agenda	
  
•  Part	
  I	
  	
  
–  Introduc-ons	
  
–  Introduc-on	
  to	
  OpenStack	
  
•  Part	
  II	
  	
  
–  Deep	
 ...
Agenda	
  
•  Part	
  I	
  	
  
–  Introduc,ons	
  
–  Introduc,on	
  to	
  OpenStack	
  
•  Part	
  II	
  	
  
–  Deep	
 ...
Introduc-on	
  
•  Me	
  
– Devda8a	
  Kulkarni	
  
– First	
  contributor	
  on	
  Solum	
  
– Currently	
  Project	
  Te...
Introduc-on	
  to	
  OpenStack	
  
What	
  is	
  OpenStack?	
  
A	
  family	
  of	
  “open”	
  projects	
  that	
  together	
  enable	
  different	
  
aspects...
OpenStack	
  family	
  
•  Family	
  of	
  projects	
  enabling	
  different	
  aspects	
  of	
  cloud	
  compu-ng	
  
–  A...
OpenStack	
  by	
  numbers	
  
•  49	
  projects	
  in	
  OpenStack	
  big	
  tent	
  
•  Several	
  hundred	
  organiza-o...
Agenda	
  
•  Part	
  I	
  	
  
–  Introduc-ons	
  
–  Introduc-on	
  to	
  OpenStack	
  
•  Part	
  II	
  	
  
–  Deep	
 ...
 	
  What	
  is	
  Solum?	
  
OpenStack	
  project	
  that	
  provides	
  easy	
  mechanism	
  for	
  applica-on	
  
devel...
 	
  Project	
  goals	
  
Applica-on	
  Stack	
  Flexibility	
  
Ability	
  to	
  support	
  applica-ons	
  wri8en	
  in	
...
 	
  Project	
  details	
  
•  Wiki:	
  
h8ps://wiki.openstack.org/wiki/Solum	
  
	
  
•  Mee-ngs:	
  
Tuesdays	
  1700	
 ...
 	
  Solum	
  abstrac-ons	
  
•  App	
  
•  Languagepack	
  (LP)	
  
•  Deployment	
  Unit	
  (DU)	
  
•  Workflow	
  
•  A...
 	
  Solum	
  abstrac-ons	
  -­‐	
  App	
  
Declara-ve	
  applica-on	
  
defini-on	
  	
  
•  source	
  code	
  repository	...
 	
  Solum	
  abstrac-ons	
  -­‐	
  Languagepack	
  (LP)	
  
•  Docker	
  image	
  which	
  contains	
  applica-on	
  spec...
 	
  Solum	
  abstrac-ons	
  -­‐	
  Languagepack	
  Example:	
  Python	
  
	
  
	
  
FROM ubuntu:precise
MAINTAINER Murali...
 	
  Solum	
  abstrac-ons	
  -­‐	
  Deployment	
  Unit	
  (DU)	
  (1/2)	
  
•  Docker	
  image	
  that	
  is	
  formed	
  ...
 	
  Solum	
  abstrac-ons	
  -­‐	
  Deployment	
  Unit	
  (DU)	
  (2/2)	
  
•  Building	
  a	
  DU	
  
–  Start	
  from	
 ...
 Solum	
  abstrac-ons	
  -­‐	
  Workflow	
  
•  Abstrac-on	
  to	
  represent	
  execu-on	
  of	
  applica-on	
  deployment...
Solum	
  abstrac-ons	
  -­‐	
  Add-­‐ons	
  
•  Services	
  needed	
  by	
  an	
  applica-on	
  
E.g.:	
  rela-onal	
  dat...
Architecture	
  and	
  system	
  details	
  	
  
 	
  High-­‐level	
  architecture	
  
Solum	
  
API	
  
Solum	
  
Worker	
  
Solum	
  
Deployer	
  Queue	
   Queue	
  
Que...
Demo	
  
h8ps://vimeo.com/143425822	
  
	
  
Agenda	
  
•  Part	
  I	
  	
  
–  Introduc-ons	
  
–  Introduc-on	
  to	
  OpenStack	
  
•  Part	
  II	
  	
  
–  Deep	
 ...
Contribu-ng	
  to	
  OpenStack	
  
Tools/Systems	
  
•  Gerrit:	
  Code	
  review	
  system	
  
–  h8ps://review.openstack.org/	
  
•  Launchpad:	
  Work	
  ...
Setup	
  steps	
  (1/2)	
  
•  h8p://docs.openstack.org/infra/manual/developers.html	
  
•  Create	
  Launchpad	
  account...
Setup	
  steps:	
  Problems	
  and	
  troubleshoo-ng	
  
•  Problem:	
  
We	
  don't	
  know	
  where	
  your	
  gerrit	
 ...
Setup	
  steps:	
  Problems	
  and	
  troubleshoo-ng	
  
•  Problem:	
  
	
  	
  Problem	
  running	
  'git	
  remote	
  u...
Steps	
  to	
  submit	
  a	
  patch	
  
•  Create	
  a	
  Launchpad	
  bug	
  or	
  assign	
  an	
  exis-ng	
  bug	
  to	
...
Steps	
  to	
  merge	
  a	
  patch	
  
•  Patch	
  needs	
  to	
  pass	
  configured	
  automated	
  tests	
  
•  Patch	
  ...
OpenStack	
  CI	
  System	
  
Con-nuous	
  Integra-on	
  of	
  patches	
  
•  Systems	
  involved:	
  
– Zuul	
  
– Gerrit	
  
– Jenkins	
  
– Github	
 ...
OpenStack’s	
  CI	
  setup	
  
Gerrit	
   Zuul	
   Jenkins	
  
Github	
  
2	
   3	
  
4	
  5	
  
1	
  
Patch	
  
submi8ed	...
Patch	
  review	
  page	
  details	
  
Gerrit:	
  Patch	
  that	
  fails	
  gate	
  tests	
  
Development	
  best	
  prac-ces	
  
Best	
  prac-ces,	
  tricks,	
  -ps	
  
•  Devstack	
  
•  Vagrant	
  and	
  Virtualbox	
  
•  Python	
  	
  
•  Screen	
 ...
Devstack	
  
•  OpenStack	
  environment	
  consis-ng	
  of	
  all	
  the	
  
openstack-­‐services	
  
– Keystone,	
  Nova...
Vagrant	
  and	
  VirtualBox	
  
•  Vagrant	
  
–  Configura-on	
  management	
  tool	
  for	
  configuring	
  virtual	
  
m...
Sexng	
  up	
  Solum	
  development	
  env	
  
•  Develop	
  on	
  Host,	
  debug	
  on	
  the	
  VM	
  
•  Install	
  Vag...
Python	
  
•  PyCharm	
  IDE	
  
•  pdb	
  
– Put	
  breakpoint	
  in	
  the	
  code	
  where	
  you	
  want	
  to	
  brea...
Tes-ng	
  
•  Run	
  tests	
  locally	
  before	
  submixng	
  patch	
  for	
  review	
  
– From	
  within	
  solum	
  dir...
Code	
  paths	
  on	
  Devstack	
  
•  solum	
  
/opt/stack/solum	
  
•  python-­‐solumclient	
  
/opt/stack/python-­‐solu...
Screen	
  
•  Program	
  that	
  allows	
  sharing	
  physical	
  terminal	
  between	
  
different	
  processes	
  
•  Ent...
Debugging	
  
•  Debug	
  code/feature	
  
– Put	
  breakpoints	
  in	
  the	
  code	
  
– Invoke	
  code-­‐under-­‐test	
...
Git	
  
•  Crea-ng	
  dependent	
  commits	
  
•  Squashing	
  commits	
  
– git	
  rebase	
  -­‐i	
  HEAD~<number-­‐of-­‐...
Hands-­‐on	
  exercise	
  
•  Submit	
  first	
  patch	
  to	
  OpenStack	
  (45	
  minutes)	
  
•  h8ps://etherpad.opensta...
Ques-ons	
  
Upcoming SlideShare
Loading in …5
×

Contributing to OpenStack

319 views

Published on

Overview of OpenStack, tools required for contributing to OpenStack, high-level introduction of Project Solum

Published in: Engineering
  • Be the first to comment

Contributing to OpenStack

  1. 1. Tutorial  on  Contribu-ng  to   OpenStack   Devda8a  Kulkarni   PTL,  OpenStack  Solum  
  2. 2. Agenda   •  Part  I     –  Introduc-ons   –  Introduc-on  to  OpenStack   •  Part  II     –  Deep  dive  of  a  OpenStack  project  -­‐  Solum   •  Part  III   –  Contribu-ng  to  OpenStack   –  OpenStack  CI  system   –  Development  best  prac-ces   –  Hands-­‐on  exercise   •  Ques-ons  and  Answers  
  3. 3. Agenda   •  Part  I     –  Introduc,ons   –  Introduc,on  to  OpenStack   •  Part  II     –  Deep  dive  of  a  OpenStack  project  -­‐  Solum   •  Part  III   –  Contribu-ng  to  OpenStack   –  OpenStack  CI  system   –  Development  best  prac-ces   –  Hands-­‐on  exercise   •  Ques-ons  and  Answers  
  4. 4. Introduc-on   •  Me   – Devda8a  Kulkarni   – First  contributor  on  Solum   – Currently  Project  Team  Lead  (PTL)  of  Solum  for  Mitaka   cycle   – PhD,  University  of  Minnesota  Minneapolis   •  You  
  5. 5. Introduc-on  to  OpenStack  
  6. 6. What  is  OpenStack?   A  family  of  “open”  projects  that  together  enable  different   aspects  of  cloud  compu-ng     Openness  of  OpenStack    h8ps://wiki.openstack.org/wiki/Open   •  Open  source   •  Open  design   •  Open  development   •  Open  community    
  7. 7. OpenStack  family   •  Family  of  projects  enabling  different  aspects  of  cloud  compu-ng   –  Authen-ca-on  and  authoriza-on  layer   •  Keystone   –  Infrastructure-­‐as-­‐a-­‐service  layer   •  Nova,  Glance   –  Storage  layer   •  Swi[,  Cinder   –  Networking  layer   •  Neutron   –  Plaorm-­‐as-­‐a-­‐service  layer   •  Solum,  Heat,  Murano,  Mistral,  Trove   •  Project  types   –  OpenStack  big  tent  vs.  OpenStack  related  projects      
  8. 8. OpenStack  by  numbers   •  49  projects  in  OpenStack  big  tent   •  Several  hundred  organiza-ons  involved     •  Several  thousand  developers  par-cipa-ng  
  9. 9. Agenda   •  Part  I     –  Introduc-ons   –  Introduc-on  to  OpenStack   •  Part  II     –  Deep  dive  of  a  OpenStack  project  -­‐  Solum   •  Part  III   –  Contribu-ng  to  OpenStack   –  OpenStack  CI  system   –  Development  best  prac-ces   –  Hands-­‐on  exercise   •  Ques-ons  and  Answers  
  10. 10.    What  is  Solum?   OpenStack  project  that  provides  easy  mechanism  for  applica-on   developers  to  deploy  and  run  their  applica-ons  on  OpenStack  star-ng   from  applica-on’s  source  code     Why  should  you  care?  (the  value  proposi-on)   –  For  operators,  Solum  provides  ability  to  make  your  OpenStack  cloud  more   useful  for  your  clouds’  applica-on  developers  by  simplifying  the  process  of   deploying  applica-ons  to  it   –  For  developers,  Solum  provides  an  easy-­‐to-­‐use  plaorm  for  building,  tes-ng,   and  deploying  applica-ons  on  OpenStack  clouds  
  11. 11.    Project  goals   Applica-on  Stack  Flexibility   Ability  to  support  applica-ons  wri8en  in  different  languages  and  using  different  applica-on   frameworks     Developer  Produc-vity   Ability  to  perform  CI/CD,  integra-on  with  github     Add-­‐On  Services  Extensibility   Ability  to  support  different  add-­‐on  services     Applica-on  Portability   Ability  to  deploy  applica-ons  across  different  OpenStack  clouds  (use  na-ve  OpenStack   services)    
  12. 12.    Project  details   •  Wiki:   h8ps://wiki.openstack.org/wiki/Solum     •  Mee-ngs:   Tuesdays  1700  UTC  in  #openstack-­‐mee-ng-­‐3   •  IRC:   Channel:  #solum  on  freenode   •  Code:   h8ps://github.com/openstack/solum   h8ps://github.com/openstack/python-­‐solumclient  
  13. 13.    Solum  abstrac-ons   •  App   •  Languagepack  (LP)   •  Deployment  Unit  (DU)   •  Workflow   •  Add-­‐on  
  14. 14.    Solum  abstrac-ons  -­‐  App   Declara-ve  applica-on   defini-on     •  source  code  repository   •  run  command   •  port   •  languagepack   version: 1 name: cherrypy description: python web app languagepack: python source: repository: https://github.com/ rackspace-solum-samples/solum- python-sample-app.git revision: master workflow_config: run_cmd: python app.py ports: - 80
  15. 15.    Solum  abstrac-ons  -­‐  Languagepack  (LP)   •  Docker  image  which  contains  applica-on  specific  build  and  run-me   libraries   •  A  LP  needs  to  implement  Solum’s  languagepack  contract    A  well-­‐known  file  available  at  a  known  loca-on  on  the  languagepack  image    /solum/bin/build.sh   •  How  to  build  a  LP?   Dockerfile     •  Operator-­‐defined  or  user-­‐defined  
  16. 16.    Solum  abstrac-ons  -­‐  Languagepack  Example:  Python       FROM ubuntu:precise MAINTAINER Murali Allada <murali.allada@rackspace.com> RUN apt-get -yqq update RUN apt-get -yqq install python-pip RUN apt-get -yqq install python-dev COPY build.sh /solum/bin/ https://github.com/rackspace-solum-samples/solum-languagepack-python   #!/bin/bash # Check if pip is installed pip help [[ $? != 0 ]] && echo python-pip is not installed. && exit 1 # Install app dependencies cd /app pip install -r requirements.txt build.shDockerfile
  17. 17.    Solum  abstrac-ons  -­‐  Deployment  Unit  (DU)  (1/2)   •  Docker  image  that  is  formed  from  the  languagepack  image  with   applica-on’s  source  code  added  to  it    DU  =  LP  +  applica-on  source  code   •  Solum  guarantees  the  DU  contract   Applica-on  source  code  available  at  a  known  loca-on  on  the  DU  image        /app     •  How  to  build  a  DU?   –  Construct  Dockerfile  with  languagepack  as  the  base  image,  applica-on  source  code   injected,  run  command  as  the  entry  point   –  Build  the  DU  image  from  this  Dockerfile  
  18. 18.    Solum  abstrac-ons  -­‐  Deployment  Unit  (DU)  (2/2)   •  Building  a  DU   –  Start  from  the  specified  languagepack  LP   –  Specify  execu-on  of  languagepack’s  ‘build.sh’   –  Inject  applica-on  source  code   –  Use  the  run  command  specified  in  app  defini-on  as  the  default  Entrypoint  to   run  the  DU   •  LP  and  DU  storage   Glance,  Swi[,  Docker  registry   18  
  19. 19.  Solum  abstrac-ons  -­‐  Workflow   •  Abstrac-on  to  represent  execu-on  of  applica-on  deployment   consis-ng  of  one  or  more  workflow  stages     •  Supported  workflow  stages   –  Build  DU  (and  store  it  for  future  use)   –  Run  unit  tests,  build  DU   –  Run  unit  tests,  build  DU,  deploy  DU  (if  unit  tests  pass)   –  Build  DU,  deploy  DU   –  Deploy  a  previously  built  DU  (not  yet  available)   •  A  workflow  can  be  triggered  from  github  webhooks    
  20. 20. Solum  abstrac-ons  -­‐  Add-­‐ons   •  Services  needed  by  an  applica-on   E.g.:  rela-onal  database  such  as  Trove     •  DU  parameters    Solum  supports  ability  to  pass  service’s  connec-on  parameters  to    applica-on  DU  
  21. 21. Architecture  and  system  details    
  22. 22.    High-­‐level  architecture   Solum   API   Solum   Worker   Solum   Deployer  Queue   Queue   Queue   Solum   Conductor   Heat   Solum  Database   Glance,  Swi[,  Docker  Registry   Builds  LP  and  DU   Deploys  DU   -­‐  Decoupled  architecture   -­‐  Asynchronous   communica-on  
  23. 23. Demo   h8ps://vimeo.com/143425822    
  24. 24. Agenda   •  Part  I     –  Introduc-ons   –  Introduc-on  to  OpenStack   •  Part  II     –  Deep  dive  of  a  OpenStack  project  -­‐  Solum   •  Part  III   –  Contribu,ng  to  OpenStack   –  OpenStack  CI  system   –  Development  best  prac,ces   –  Hands-­‐on  exercise   •  Ques-ons  and  Answers  
  25. 25. Contribu-ng  to  OpenStack  
  26. 26. Tools/Systems   •  Gerrit:  Code  review  system   –  h8ps://review.openstack.org/   •  Launchpad:  Work  (bugs/blueprints)  tracking  system   –  h8ps://launchpad.net/   •  Github:  Code  repository   –  h8ps://github.com/openstack   •  Git:  Code  contribu-on   •  Mailing  list:  openstack-­‐dev   –  h8p://lists.openstack.org/cgi-­‐bin/mailman/lis-nfo/openstack-­‐dev   •  IRC:  chat.freenode.net  
  27. 27. Setup  steps  (1/2)   •  h8p://docs.openstack.org/infra/manual/developers.html   •  Create  Launchpad  account   •  Join  OpenStack  founda-on  as  founda-on  member   •  Sign  individual  contributor’s  license  agreement   •  Install  git-­‐review   •  Install  git   •  Run  git  review  -­‐s  
  28. 28. Setup  steps:  Problems  and  troubleshoo-ng   •  Problem:   We  don't  know  where  your  gerrit  is.  Please  manually   create  a  remote   •  Solu-on:   – Create  a  username  in  gerrit  if  not  already  created   – Manually  set  gerrit  remote.  E.g.:          git  remote  add  gerrit  "ssh:// djkulkarni@review.openstack.org:29418/openstack/ solum.git”  
  29. 29. Setup  steps:  Problems  and  troubleshoo-ng   •  Problem:      Problem  running  'git  remote  update  gerrit’                Fetching  gerrit                  Permission  denied  (publickey).                  fatal:  Could  not  read  from  remote  repository.   •  Solu-on:   –  Generate  sshkey  and  add  it  to  gerrit   –  ssh-­‐keygen  
  30. 30. Steps  to  submit  a  patch   •  Create  a  Launchpad  bug  or  assign  an  exis-ng  bug  to   yourself   •  Fixing  the  bug   –  Itera-ve  process   •  Understand  the  issue  (be  in  touch  with  project  developers  on   project’s  IRC  channel  or  on  openstack-­‐dev  mailing  list)   •  Design  solu-on   •  Write  code   •  Write  tests   •  Commit  code   •  Provide  good  commit  message   •  Submit  patch  for  review  
  31. 31. Steps  to  merge  a  patch   •  Patch  needs  to  pass  configured  automated  tests   •  Patch  needs  to  get  two  +2  votes   •  Patch  needs  to  be  approved  for  merging  by   project’s  core  reviewer  
  32. 32. OpenStack  CI  System  
  33. 33. Con-nuous  Integra-on  of  patches   •  Systems  involved:   – Zuul   – Gerrit   – Jenkins   – Github   •  Pre-­‐merge  checking   – Patch  is  merged  to  master  only  if  all  the  configured  tests   pass  
  34. 34. OpenStack’s  CI  setup   Gerrit   Zuul   Jenkins   Github   2   3   4  5   1   Patch   submi8ed   for  review   Patch  picked  up   for  tes-ng   Tests  ran  on  the  patch   Jenkins  gives  its  vote   Code   merged  
  35. 35. Patch  review  page  details  
  36. 36. Gerrit:  Patch  that  fails  gate  tests  
  37. 37. Development  best  prac-ces  
  38. 38. Best  prac-ces,  tricks,  -ps   •  Devstack   •  Vagrant  and  Virtualbox   •  Python     •  Screen   •  Debugging   •  Git  
  39. 39. Devstack   •  OpenStack  environment  consis-ng  of  all  the   openstack-­‐services   – Keystone,  Nova,  Heat,  Solum,  etc.   •  Can  be  run  within  a  virtual  machine  or   independently   •  Useful  for  development  
  40. 40. Vagrant  and  VirtualBox   •  Vagrant   –  Configura-on  management  tool  for  configuring  virtual   machines   •  VirtualBox   –  Virtualiza-on  system   •  Spinning  up  a  development  environment  using  Vagrant   +  VirtualBox   –  h8ps://github.com/rackerlabs/vagrant-­‐solum-­‐dev   –  git  clone  <solum-­‐vagrant-­‐repo>   –  vagrant  up  -­‐-­‐provision  
  41. 41. Sexng  up  Solum  development  env   •  Develop  on  Host,  debug  on  the  VM   •  Install  Vagrant  and  Virtualbox   •  Clone  solum  and  python-­‐solumclient  repositories  on   the  Host   •  Set  following  environment  variables:   –  SOLUM=<path  on  Host  system  to  solum  repo>   –  SOLUMCLIENT=<path  on  Host  system  to  python-­‐solumclient   repo>   •  vagrant  up  -­‐-­‐provision   •  vagrant  ssh  
  42. 42. Python   •  PyCharm  IDE   •  pdb   – Put  breakpoint  in  the  code  where  you  want  to  break   •  import  pdb;  pdb.set_trace()   – pdb  commands   •  s:  Execute  the  current  line,  stop  at  first  possible  occasion   •  c:  Con-nue  execu-on,  only  stop  when  a  breakpoint  is   encountered   •  h8ps://docs.python.org/2/library/pdb.html  
  43. 43. Tes-ng   •  Run  tests  locally  before  submixng  patch  for  review   – From  within  solum  directory   •  tox  –epep8   •  tox  –epy27   •  Tox   – Command  line  tool  for  running  tests  within  virtualenv  
  44. 44. Code  paths  on  Devstack   •  solum   /opt/stack/solum   •  python-­‐solumclient   /opt/stack/python-­‐solumclient   •  Solum  logs   –  /var/log/solum/deployer   –  /var/log/solum/worker   •  Devstack  logs  (of  various  screen  sessions)   –  /opt/stack/logs/  
  45. 45. Screen   •  Program  that  allows  sharing  physical  terminal  between   different  processes   •  Entering  Solum’s  screen  session  in  Devstack   –  screen  –x  stack   •  Screen  commands   –  Move  to  next  window:  Ctrl+a  n   –  Move  to  previous  window:  Ctrl+a  p   –  See  all  windows:  Ctrl+a  “   –  Enter  scrollback/copy  mode:  Ctrl+a  [   –  Leave  scrollback/copy  mode:  Ctrl+a  ]   –  h8p://www.pixelbeat.org/lkdb/screen.html  
  46. 46. Debugging   •  Debug  code/feature   – Put  breakpoints  in  the  code   – Invoke  code-­‐under-­‐test  from  CLI   – Use  screen  to  enter  breakpoint  and  then  debug  using   Pdb   •  Debug  failing  func-onal  test   – h8ps://review.openstack.org/#/c/254344/   •  Debug  failing  unit  test  
  47. 47. Git   •  Crea-ng  dependent  commits   •  Squashing  commits   – git  rebase  -­‐i  HEAD~<number-­‐of-­‐commits>   – pick/squash/edit   •  Upda-ng  dependent  commit  and  submixng  for   review   •  h8ps://ask.openstack.org/en/ques-on/31633/ gerrit-­‐best-­‐way-­‐to-­‐make-­‐a-­‐series-­‐of-­‐dependent-­‐ commits/  
  48. 48. Hands-­‐on  exercise   •  Submit  first  patch  to  OpenStack  (45  minutes)   •  h8ps://etherpad.openstack.org/p/solum-­‐low-­‐ hanging-­‐fruit-­‐bugs  
  49. 49. Ques-ons  

×