Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

WSO2Con EU 2015: Open Source Journey at Ordnance Survey


Published on

WSO2Con EU 2015: Open Source Journey at Ordnance Survey

When five years ago the outgoing Minister of Finance Liam Burne left a note at the treasury that said “there’s no cash left” it concentrated minds in the Civil Service and an ‘Open Source First’ policy was born. This led to a successful WSO2 engagement, the proliferation of open source solutions at OS, and ultimately a significant saving in software costs. This session will discusses the highlights and pitfalls along the way.


Hillary Corney
Principle Architect,
Ordnance Survey

Published in: Technology
  • Be the first to comment

WSO2Con EU 2015: Open Source Journey at Ordnance Survey

  1. 1. The  Open  Source   Journey  at  OS   Hillary  Corney   Principal  Architect   Ordnance  Survey    
  2. 2. An  introduc9on  to  Ordnance  Survey   •  Ordnance  Survey  over  220  years  old.  Founded  in  1791   •  Started  as  a  military  mapping  unit  now  the  UK’s   na9onal  mapping  agency.   •  No  longer  military,  now  a  UK  Government  owned   company  a  “GovCo”   •  Famous  for  the  best  paper  maps  in  the  world   •  What  you  may  not  know  is  that  OS  is  now  a  mostly   digital  organisa9on  and  that  it  hosts  the  largest   Geospa9al  database  in  the  world  called  “MasterMap”.   •  We  also  hold  the  Guinness  Book  of  Records  world   record  for  the  largest  MinecraT  map  ever  made  with   an  astonishing  83  billion  blocks.  (And  you  thought  big     Government  compu9ng  projects  always  failed!)  
  3. 3. Digital  Maps  of  the  UK  accurate  to  a   1cm  updated  10,000  9mes  a  day    
  4. 4. Produc9on  IT  soTware  at  Ordnance  Survey   •  Ini9al  experiments  using  IT  to  support  map  making  started  in  late  1960s   •  First  digital  product  made  available  in  1971  (1:125,000  coastline)   •  Increasing  ramping  up  of  digi9sa9on  (1970s  –  1990s)   •  Move  from  automa9ng  map  produc9on  to  digital  data  (1990s  onwards)   •  In  the  early  years  much  of  the  Mapping    soTware  we  used  was  bespoke   •  Over  9me  increasing  use  made  of  proprietary  off-­‐the-­‐shelf  soTware  products  (as   capability  aligned  with  our  needs)   •  In  the  last  10  years,  open-­‐source  op9ons  have  developed  and  matured  (as   capability  has  improved)   Bespoke   Proprietary   Open  source   1970   Mid-­‐1980s   Mid-­‐2000s   Today  1970   Mid-­‐1980s   Mid-­‐2000s   Today  
  5. 5. The  state  of  Open  Source  SoTware   adop9on  at  OS  Pre-­‐2011   •  Managers  and  Architects  shied  away  from  open   source.   •  Its  use  was  organic,  developer  led,  no  enterprise   level  involvement  or  strategy   •  Open  Source  soTware  used  in  ad-­‐hoc  way   •  OTen  used  “under  the  radar”   •  SoTware  repositories  and  version  control   available  but  seldom  used.   •  Very  licle  maintenance,  Patching,  upgrades  etc.   •  Separate  discrete  code  bases  for  the  same   soTware   •  Almost  no  hand-­‐over  into  produc9on  and  licle   documenta9on    
  6. 6. A  short  history  of  open  source  at  Ordnance  Survey   Apache, Tomcat, J2EE & MySQL for initial web systems 2002 Linux operating system for web- facing applications 2004 Linux operating system for internal production systems 2006 Using Open Layers for OS OpenSpace 2008 PostGIS, GeoNetwork, GeoServer, INSPIRE 2010 PostGIS, Web Services Consolidation 2012 2013 Magento, Solr for Map Shop, Apache Jena for linked data Open source First
  7. 7. The  Open  Source  in  the  Enterprise   “revolu9on”  at  OS   •  In  2010  the  outgoing  Treasury  Minister   Liam  Burne,  leT  a  now  infamous  note,   “there's  no  cash  le.”.  He  has  recently  said   it  was  a  foolish  thing  to  do  and  not  a  day   goes  by  without  him  feeling  guilty  for   wri9ng  it.   •  But  don't  despair  Liam,  there  was  an   unexpected  upside......   •  The  incoming  government,  worried,  and   looking  for  savings,  sent  out  the  “Open   Source  first”  direc9ve.  
  8. 8. The key points of the Government’s new directive were: 1)  The  Government  will  ac=vely  and  fairly  consider  open  source   solu=ons  alongside  proprietary  ones  in  making  procurement   decisions,     2)  Procurement  decisions  will  be  made  on  the  basis  on  the  best   value  for  money  solu9on  to  the  business  requirement,  taking   account  of  total  life9me  cost  of  ownership  of  the  solu9on,   including  exit  and  transi9on  costs,  aTer  ensuring  that  solu9ons   fulfill  minimum  and  essen9al  capability,  security,  scalability,   transferability,  support  and  manageability  requirements.   3)  The  Government  will  expect  those  pujng  forward  IT  solu9ons   to  develop  where  necessary  a  suitable  mix  of  open  source  and   proprietary  products  to  ensure  that  the  best  possible  overall   solu9on  can  be  considered.     4)  Where  there  is  no  significant  overall  cost  difference  between   open  and  non-­‐open  source  products,  open  source  will  be   selected  on  the  basis  of  its  addi=onal  inherent  flexibility.    
  9. 9. How  do  you  “ac=vely  and  fairly  consider   open  source  solu=ons  alongside   proprietary  ones”?   •  What  are  the  barriers  to  the  adop=on  of  Open   Source  So.ware?   •  The  lack  of  a  Vendor  is  a  factor  in  low  OSS   uptake.   •  Because  there  is  no  Vendor:   •  No  one  promotes  and  markets    the  soTware   •  No  one  pays  for  reviews  of  the  soTware   •  No  one  champions  the  soTware   •  No  one  sponsors  the  soTware   •   So  OS  so.ware  is  o.en  invisible,  even  good  OS   so.ware  is  o.en  a  well  kept  secret.      
  10. 10. Barriers  to  the  adop9on  of  Open  Source   Software: •  Myths  and  FUD  taken  as  truth.   •  MicrosoT  and  others  spent  large  amounts  of  effort  in   the  first  half  of  the  last  decade  debunking  Linux/OSS  by   spreading  fear  and  rumours  which  they  now  admit   actually  backfired  and  increased  awareness  and   adop9on  of  Linux/OSS  but  much  of  the  misinforma9on   from  those  days  s9ll  lingers  on.   •   If  you  are  interested  Google  “MicrosoT  The  Halloween   document”   •  Lack  of  a  “Vendor”  driving  any  tender  response   •  Lack  of  external  support  Lack  of  documenta9on   •  Lack  of  appropriate  in-­‐house  coding  skills   •  Lack  of  fair  and  honest  evalua=ons  comparisons   and  reviews  
  11. 11. More  Myths  -­‐  Open  Source  SoTware   is  “poor  quality”   •  A  Coverity  2011  survey,  found  Open  source  was  as  good  as   proprietary  soTware   •  “Both  open  source  code  quality  and  proprietary  code   quality,  as  measured  by  defect  density,  is  becer  than  the   average  for  the  soTware  industry,  which  is  a  defect  density   of  1.0.”   •  “Linux  2.6,  PHP  5.3,  and  PostgreSQL  9.1  are  recognized  as   open  source  projects  with  superior  code  quality  and  can  be   used  as  industry  benchmarks,  achieving  defect  densi9es  of  . 62,  .20,  and  .21  respec9vely.”   •  Open  source  code  quality  is  on  par  with  proprietary   code  quality,  par9cularly  in  cases  where  codebases   are  of  similar  size.  For  instance,  Linux  2.6,  a  project   with  nearly  7  million  lines  of  code,  has  a  defect  density   of  .62  which  is  roughly  iden9cal  to  that  of  its   proprietary  codebase  counterparts.  
  12. 12. Coverity's  2013  Scan  concluded  that  the   defect  density  of  Open  Source  code  was   now    be-er  than  proprietary  code!   •  In  2013,  the  defect  density  rate  of  open   source  code  was  lower  than  that  of   proprietary  code.  According  to  recent   reports  sponsored  by  Black  Duck  SoTware   and  North  Bridge  Venture  Partners,   “quality  concerns  are  no  longer  a  barrier  to   open  source  adop9on  in  the  enterprise.  In   fact,  the  quality  of  the  open  source  code   for  Coverity  Scan  par9cipants  can  be   higher  than  the  proprietary  code  included   in  an  enterprise  product.”  
  13. 13. So  if  quality  is  not  a  problem  how  do  you  “ac=vely   and  fairly  consider  open  source  solu=ons   alongside  proprietary  ones”?     •  Appoint  an  internal  Open  Source   Champion.  Someone  who  will  take  the   place  of  the  missing  vendor  and  “market”   the  product  to  the  stakeholders.   •  Use  proof  of  concept  builds  to  evaluate   the  suitability  of  OSS.   •  Think  crea9vely  about  how  you  can  find   equivalents  to  the  normal  Vendor   procurement  process.  
  14. 14. Evaluate OSS like any COTS package but…. •  Where  you  would  evaluate  the  vendor  subs9tute   the  User  community.   •  Where  you  would  evaluate  help  desk  support   look  at  the  availability  of  of  OSS  Specialist  Books,   the  community  and  internal  knowledge.     •  Where  you  would  review  the  release  roadmap   Look  at  the  release  frequency  and  access  to   nightly  builds.   •  Ask  the  user  community  for  reference  cases  and   recommenda9ons.  
  15. 15. Understand  Licensing   •  There  are  over  70  licenses  approved  by  the  open  source   ini=a=ve,  so  it  is  definitely  not  an  easy  task  to  understand   all  of  the  licenses  out  there.   •   Fortunately  the  10  most  popular  licenses  make  up  about   80%  of  the  open  source  available,  so  understanding  the   first  10  licenses  is  a  preXy  reasonable  start.     •  Create  an  open  source  review  board  consis9ng  of  both   legal  counsel  and  SoTware  engineering  governance,  who   can  understand  the  implica9ons  of  the  various  licenses.     •  Create  a  summary  of  current  OS  licences  keep  it  up  to   date.   •  Lead  developers  and  Architects  will  need  to  review  the   summary  and  be  aware  of  the  legal  traps.  
  16. 16. Working  with  Open  Source   •  As Laurie Wurster of the Gartner Group said: “Just because something is free, doesn’t mean it has no cost.” “Companies must have a policy for procuring OSS, deciding which applications will be supported by OSS, and identifying the intellectual property risk or supportability risk associated with using OSS. Once a policy is in place, then there must be a governance process to enforce it.” •  There must be a policy and enforcing governance for procuring Open source. •  There must also be a policy and governance for implementing, developing and supporting Open Source Software.
  17. 17. Recommenda9ons   •  We  incorporate  the  Government  Open  Source  Policy  into  our   procurement  policy   •  Ensure  that  when  Evalua9ng  OSS  there  is  a  quick  route  to  financing   proof  of  Concept  Evalua9on  Environments.   •  We  adopt  a  structured  approach  to  evalua9ng  Open  Source   SoTware.   •  We  educate  developers  to  be  aware  of  the  risks  inherent  in  Open   Source  licences.  We  encourage  open  source  skills  in  house  and  we   create  a  library  of  open  source  books  for  developers  to  use.   •  We  appoint  OSS  Champions  to  each  Procurement  ini9a9ve  to  make   sure  that  OSS  is  considered  fairly  and  is  “Marketed”  to  the   Procurement  Stakeholders  in  a  proac9ve  way.   •  We  ensure  good  legal  and  technical  governance  of  our  use  of  Open   Source.   •  We  create  our  own  Ordnance  Survey  Open  Source  Licence  
  18. 18. Devise  a   “Procurement   decision  chart”  
  19. 19. Open  Source  first   Based  on  our  experiences,  we  have  adopted     an  “open  source  first”  enterprise  level  policy  for  new  soTware     When  looking  for  new  soTware  we  will  evaluate  open  source  packages   ahead  of  proprietary  ones   We  will  adopt  an  open  source  solu9on  if:   •  The  licence  is  suitable  for  our  needs   •  The  project  is  well  supported  with  a  lively  and  responsive  user   base   •  The  documenta9on  is  good   •  We  have  the  appropriate  skills  in-­‐house  or  training  is  readily   available   •  We  can  support  it  (or  we  can  buy  support)   •  It  works  equally  well  as  available  proprietary  soTware.     If  not,  we  will  go  through  standard  COTS  selec9on  /  procurement   processes    
  20. 20. What  have  we  learned   •  Start  small,  download  the  soTware  and  try   stuff  out.   •  Involve  and  listen  to  your  developers   •  Broaden  technical  knowledge  and  skills   •  invest  in  training   •  Change  the  COTS  only  culture   •  “Open  Source”  is  free  is  a  myth  the  costs  are   just  in  different  places.   •  Be  rigorous  about  evalua9ng  licenses    
  21. 21. What  about  support?   •  Just  because  it’s  Open  Source  does  not   necessarily  mean  you  are  on  your  own,  but   the  support  model  is  different.   •  You  can  do  it  in  house  (but  be  aware  of  the   effort  involved  and  don’t  scrimp)   •  Or  you  can  buy  support  from  vendors  such   as  WSO2  which  is  an  increasing  sign  that   Open  Source  is  entering  the  mainstream.  
  22. 22. How  did  we  apply  what  we  have  learned  to   our  API  delivery  plarorm  project?   •  We  reviewed  the  Open  Source  packages  first.   •  We  evaluated  WSO2  against  other  Open   Source  and  COTS  solu9ons     •  We  loaded  it  onto  our  developer  sandboxes   and  “kicked  the  9res”   •  We  engaged  WSO2  and  ini9ated  a  very   successful  proof  of  concept….  
  23. 23. POC  Solu9on  hosted  on  two  servers   in  AWS  and  built  with  a  Joint  OS/ WSO2  team  in  just  two  weeks  
  24. 24. Integra9on  with  Magento   •  Another  benefit  of  the  “Open  Source  First”   approach  is  that  we  are  able  to  closely  couple   and  integrate  another  Open  Source  program   “Magento  Enterprise”  a  leading  eCommerce   package  with  the  WSO2  SoTware   •  As  we  have  access  to  the  source  code  of  both   plarorms  we  can  9ghtly  integrate  the  two  in   a  way  that  would  not  be  possible,  or  very   much  more  difficult  with  COTTS  soTware.    
  25. 25. Logical  Architecture  PRODUCTION   environment  
  26. 26. Authen9ca9on  Architecture  Integra9on  between   Magento  and  WSO2  Iden9ty  Server  via  SAML  and   SCIM  
  27. 27. The  outcome  of  the  Open  Source  First   process:   •  We  purchased  support  and  finally  having   sa9sfied  ourselves  that  WSO2  can  meet  our   needs….   •  We  engaged  WSO2  and  are  building  not  one  but   two  API  delivery  produc9on  environments  with   WSO2  soTware.  U9lising:   •  WSO2  API  manager   •  WSO2  Iden9ty  Manager   •  WSO2  Business  ac9vity  Monitor   •  WSO2  Complex  event  processor   •  Magento  Enterprise  Open  Source  Shopping  cart.  
  28. 28. The  best  of  both  worlds?   •  It  is  my  opinion  that  organisa9ons  like  WSO2  and   Magento  offer  the  best  of  both  worlds.   •  A  vibrant  user  community  feeding  ideas  and  fixes   back  to  the  user  base   •  With  Enterprise  level  support  and  SoTware   roadmap  and  strategy  from  the  Parent   organisa9on.   •  WSO2  are  the  “Vendor”  providing  Enterprise   level  support  oTen  missing  from  tradi9onal  Open   Source  products,  eBay  Enterprise  provide  similar   Support  for  Magento.  
  29. 29. Ian  James  OS  Chief  Architect  said:   “Open  Source  allows  us  to  deliver  more   effec9vely”     “Open  Source  is  mainstream”     “Open  Source  is  here  to  stay  at  Ordnance   Survey”  
  30. 30. Thank  You   Ques9ons?