Enterprise	  OSGi	  at	  eBay	  	  Tony	  Ng	  Director	  of	  Engineering	  May,	  2012	  
Agenda	  •  About	  eBay	  •  Why	  OSGi	  •  How	  are	  we	  using	  OSGi	  •  Lessons	  learned	                       ...
$68.6billionIn merchandise sold in 2011                              3	  
eBay	  at	  Internet	  Scale	  •  100+	  million	  acHve	  buyers	  and	  sellers	  worldwide	  •  250	  million	  queries...
Challenges	  for	  our	  Code	  Base	  •  ~	  44	  million	  of	  lines	  of	  code	  and	     growing	         –  O(1,000...
Next	  Gen	  Pla>orm	  Guiding	  Principles	  •  Modularity	  is	  first	  class	  design	  concept	  •  Technology	  refre...
Next	  Gen	  Stack	                                             Applications (ebay.com, Mobile)                           ...
Our	  Goals	  with	  Modularity	  •  Tame	  complexity	  	  •  Organize	  our	  code	  base	  in	  loose	  coupling	  fash...
Why	  OSGi	  Goal	                                                              How	  OSGi	  Helps	  Hide	  implementaEon	...
Top	  Level	  ConsideraEons	  •  What	  OSGi	  features	  to	  use	     –  Focus	  on	  modularity	  •  End-­‐to-­‐end	  d...
Bundle	  Guidelines	  •  Create	  from	  boaom	  up	  •  Sound	  architecture	  boundary	  •  Free	  of	  split	  packages...
Dynamic	  Classloading	  •  ProblemaHc	  usage	     –  Class.forName()	     –  ClassLoader	  loadClass	  or	  getResource	...
Bundle	  CreaEon	    POM               Maven    Bundle (Jar)                        with            Global                ...
OSGi	  Manifest	  GeneraEon	  •  Maven	  arHfact	  version	  ==	  bundle	  version	  ==	  export	     packages	  version	 ...
Two-­‐Stage	  Dependency	  Wiring	    Maven Repo                                                                       OSG...
App	  Development	  Environment	  •  Eclipse	  Helios	     –  Geronimo	  Eclipse	  Plugin	     –  Apache	  Felix	  Maven	 ...
ApplicaEon	  Development	  •  ApplicaHons	  packaged	  as	  EBA	  •  Build	  Hme:	  Plajorm	  parent	  POM	     –  Pre-­‐d...
ApplicaEon	  Deployment	  EBAApplication WAR                                          VM / PhysicalAll dependent bundles  ...
ExisEng	  Code	  MigraEon	  •  Incrementally	  migrate	  applicaHons	  to	  new	  stack	  •  Most	  efforts	  spent	  in	  ...
Issues	  We	  Encountered	  •  Steep	  learning	  curve	  for	  developers	  •  Build	  tools	  integraHon	  •  Debugging	...
What	  Works	  Well	  •  OSGi	  modularity	  enforcement	  leads	  to	  beaer	  code	     organizaHon	  and	  reuse	  •  E...
Lessons	  Learned	  •  Need	  a	  few	  in-­‐house	  OSGi	  experts	         –  Evangelism	         –  Best	  pracHces	   ...
Enterprise OSGi at eBay
Upcoming SlideShare
Loading in …5
×

Enterprise OSGi at eBay

2,716 views

Published on

Published in: Technology, Education
  • Hi Tony, does the OSGi (Enterprise) works for your project? Recently, i'm working on web project which tries to involve OSGi. During the investigation, i found that integration with other Frameworks (Spring MVC, Data, Security.. MQ) is really difficult. Is it a good option to use OSGi or just use Spring framwork? Do you have suggestions? thanks.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Enterprise OSGi at eBay

  1. 1. Enterprise  OSGi  at  eBay    Tony  Ng  Director  of  Engineering  May,  2012  
  2. 2. Agenda  •  About  eBay  •  Why  OSGi  •  How  are  we  using  OSGi  •  Lessons  learned   2  
  3. 3. $68.6billionIn merchandise sold in 2011 3  
  4. 4. eBay  at  Internet  Scale  •  100+  million  acHve  buyers  and  sellers  worldwide  •  250  million  queries  each  day  to  our  search  engine  •  300+  million  items  in  50,000  categories  •  39  countries  and  10  languages    •  9  petabytes  of  data  •  2  billion  page  views  each  day  •  75  billion  database  calls   4  
  5. 5. Challenges  for  our  Code  Base  •  ~  44  million  of  lines  of  code  and   growing   –  O(1,000)  of  jars   –  O(10,000)  of  Java  packages   knowledge complexity –  O(100,000)  of  Java  classes  •  Code  is  monolithic  -­‐  many   dependencies  and  Hght  coupling  •  Simple  code  change  ripples   throughout  the  system   code size•  Developers  slow  down  and   become  risk  averse   5  
  6. 6. Next  Gen  Pla>orm  Guiding  Principles  •  Modularity  is  first  class  design  concept  •  Technology  refresh:  prefer  open  source  technologies  •  Easy  integraHon  of  mulHple  applicaHon  frameworks   and  libraries  •  Fast  development  iteraHons  •  Team  autonomy  •  OpHmize  end-­‐to-­‐end  development  cycle   6  
  7. 7. Next  Gen  Stack   Applications (ebay.com, Mobile) E-commerce Services (Search, Item, Cart, Checkout etc) Maven / Eclipse IDE Security Resource Metadata Logging Experimentation Config Localization Tracking Data Access Svcs Fwk Orchestration JS Fwk Cloud HTML5/CSS Spring MVC JQuery JSP OSGi Spring DI Servlet Init / Handler Container (Geronimo / WAS CE) JVM Operating System (Linux) Core platform Platform frameworks Application & services
  8. 8. Our  Goals  with  Modularity  •  Tame  complexity    •  Organize  our  code  base  in  loose  coupling  fashion  •  Establish  clear  code  ownership,  boundaries  and   dependencies  •  Allow  different  components  (and  teams)  evolve  at   different  speeds  •  Increase  development  agility   8
  9. 9. Why  OSGi  Goal   How  OSGi  Helps  Hide  implementaEon  details   Export-­‐Package  Accurate  understanding  and  enforcement  of   Bundle  manifest  dependencies   (Imports  &  Exports)   SemanEc  versioning  Faster  iteraEon  for  code  change   Update  and  restart   affected  bundles  only  Isolate  and  run  mulEple  versions  of  the  same   Support  for  mulEple  modules   versions  Technology  maturity   Robust  community  and   support   9  
  10. 10. Top  Level  ConsideraEons  •  What  OSGi  features  to  use   –  Focus  on  modularity  •  End-­‐to-­‐end  development  lifecycle   –  IDE  and  non-­‐IDE  paths   –  Build,  repository  and  tool  integraHon  •  ExisHng  code  migraHon  •  Developer  learning  curve   10  
  11. 11. Bundle  Guidelines  •  Create  from  boaom  up  •  Sound  architecture  boundary  •  Free  of  split  packages  •  SemanHc  versioning  •  Export  package  versions  should  match  bundle  version  •  Use  Import-­‐Package  rather  than  Require-­‐Bundle   –  With  sensible  version  ranges  •  Address  dynamic  classloading   11  
  12. 12. Dynamic  Classloading  •  ProblemaHc  usage   –  Class.forName()   –  ClassLoader  loadClass  or  getResource   –  Thread  context  class  loader  •  Short-­‐term  fix   –  DynamicImport-­‐Package:  *  •  Long-­‐term  fix   –  Refactor  code  (if  possible)  so  client  can  pass  class  or   classloader  to  framework   12  
  13. 13. Bundle  CreaEon   POM Maven Bundle (Jar) with Global OSGi Maven Bundle Manifest Repository Java source Plugin files 13  
  14. 14. OSGi  Manifest  GeneraEon  •  Maven  arHfact  version  ==  bundle  version  ==  export   packages  version  •  Follow  semanHc  versioning  (semver.org)  •  Scan  code  (BND)  to  derive  import  and  export  packages  •  Use  major  version  matching  for  Import-­‐Packages:   –  Ex:  [1.1.2,2.0)  •  By  default,  all  packages  are  exported  except   –  impl.*  and  internal.*   14  
  15. 15. Two-­‐Stage  Dependency  Wiring   Maven Repo OSGi wiring A.jar; v1 Maven A.jar; v1 based on A.jar; v2 A.jar; v2 bundle B.jar; v1 B.jar; v2 subset B.jar; v2 POM C.jar; v3 Dependency a.*; v2 . b.*; v2 . .(all bundles, all Bundle Subset Runtimeversions) •  Benefits   –  Smaller  bundle  set   –  IsolaHon  from  changes  in  global  repository  (reproducibility)   15  
  16. 16. App  Development  Environment  •  Eclipse  Helios   –  Geronimo  Eclipse  Plugin   –  Apache  Felix  Maven  Bundle  Plugin   –  OSGi  tooling  plugin   –  Standard  plugins  for  Maven  and  Git  integraHon  •  Others   –  Maven  archetypes  for  app  /  bundle  project  creaHon   –  Jenkins  CI   16  
  17. 17. ApplicaEon  Development  •  ApplicaHons  packaged  as  EBA  •  Build  Hme:  Plajorm  parent  POM   –  Pre-­‐defined  plajorm  bundles  dependencies.  Updated   regularly.   –  App  developers  can  extend  /  override  •  RaHonale   –  CerHficaHon  across  the  specific  versions   –  Ensure  applicaHon  progresses  towards  newer  versions   –  SHll  maintain  ability  to  override  on  a  per  app  basis   17  
  18. 18. ApplicaEon  Deployment  EBAApplication WAR VM / PhysicalAll dependent bundles HostLocal OBR xmlActivation scripts QA Cloud DeploymentConfiguration Files Infrastructure VM / Physical HostContainerJVM VM / Physical HostApp Deployment Unit Production 18  
  19. 19. ExisEng  Code  MigraEon  •  Incrementally  migrate  applicaHons  to  new  stack  •  Most  efforts  spent  in  migraHng  exisHng  library  jars  •  Built  tool  to  detect  duplicate  class  /  split  packages  across   jars   –  Merge  the  jars   –  Manual  refactoring  •  Deal  with  class  loading  code  •  Ensure  transiHve  dependencies  are  “bundle-­‐ized”  •  Need  to  migrate  third-­‐party  libraries  if  they  are  not  OSGi-­‐ enabled   19  
  20. 20. Issues  We  Encountered  •  Steep  learning  curve  for  developers  •  Build  tools  integraHon  •  Debugging  resoluHon  problems  •  IniHalizaHon  ordering  of  bundles  •  ImporHng  bundle  containing  only  non-­‐Java  resources  •  Bundle  space  isolaHon  between  container  and   applicaHons     20  
  21. 21. What  Works  Well  •  OSGi  modularity  enforcement  leads  to  beaer  code   organizaHon  and  reuse  •  Explicit  versioning  and  dependency  management  •  Solid  end-­‐to-­‐end  development  experience  with   proper  tooling  integraHon  •  Bundles  working  in  both  OSGi  and  non-­‐OSGi   environments  •  Dynamic  updates   21  
  22. 22. Lessons  Learned  •  Need  a  few  in-­‐house  OSGi  experts   –  Evangelism   –  Best  pracHces   –  Support  •  Reduce  direct  exposure  to  OSGi  API  for  developers  •  Important  to  prescribe  end-­‐to-­‐end  development  lifecycle  •  Some  automaHon  to  support  migraHon  of  exisHng  code   base  •  Take  small  steps   22  

×