My experience with embedding PostgreSQL

8,080
-1

Published on

At my current company, we embed PostgreSQL based technologies in various applications shipped as shrink-wrapped software. In this session we talk about the experience of embedding PostgreSQL where it is not directly exposed to end-user and the issues encountered on how they were resolved.

We will talk about business reasons,technical architecture of deployments, upgrades, security processes on how to work with embedded PostgreSQL databases.

Published in: Technology

My experience with embedding PostgreSQL

  1. 1. My  experience  with  embedding   PostgreSQL   Postgres  Conf  EU,  Dublin,  Ireland   October  2013     Jignesh  Shah,     Sr  Manager,  VMware   © 2013 VMware, Inc. All rights reserved
  2. 2. About  me   •  Sr  Manager,  VMware      Data   –  Leads  Postgres  team  at  VMware   –  Blog  at  hKp://jkshah.blogspot.com     •  Previously  Staff  Engineer,  Sun  Microsystems   –  Team  Member  of  the  first  published  SpecJAppServer   2004  benchmark  with  PostgreSQL   –  Performance  of  PostgreSQL  on  Solaris/Sun  systems   •  Working  with  PostgreSQL  circa  2005   © 2013 VMware, Inc. All rights reserved
  3. 3. Agenda   •  Business  reasons  for  embedding  database   •  Technical  architecture  journey   © 2013 VMware, Inc. All rights reserved
  4. 4. The  IniXal  Problem   •  >30    #  products  and  product  components   •  >20      requires    DB   •  Customers  have  to  get  their  own  DB  license     •  Increases  the  Total  Cost  of  Deployment   –  As  database  load  increases,  CPU  cores  required   increases,  so  does  the  total  license  cost  of  DB   © 2013 VMware, Inc. All rights reserved
  5. 5. SQL  Server  Express   •  Currently  shipped  for  some  Windows  based   products   –  However  limits  not  enough     –  Max  10GB  database  usage   •  Standard  and  Enterprise  EdiXons  get   expensive  preKy  quickly   •  Not  Available  on  Linux     © 2013 VMware, Inc. All rights reserved
  6. 6. Proposal:  PostgreSQL?   •  PostgreSQL  is  free  and  open  sourced   •  Long  history  of  stability   •  Friendly  BSD  License   •  Plus  I  am  at  a  Postgres  Conference  J   © 2013 VMware, Inc. All rights reserved
  7. 7. IniXal  External  Feedback   •  Actually  got  pushback  from  customers,  partners   –  PostgreSQL  not  part  of  their  database  standards   –  No  in-­‐house  PostgreSQL  DBA  skills   –  Lack  of  integrated  tools  ,  monitor,  ease-­‐of-­‐use   –  Large  enterprises  have  to  convince  their  managed   hosXng  partners  to  support  the  database  also   •  AKendees  here  are  going  to  be  educated  about  it     –  but  we  are  worried  about  the  rest  of  the  world   © 2013 VMware, Inc. All rights reserved
  8. 8. IniXal  Internal  Feedback     •  DB  Support  Matrix  complexity  increases   –  Every  new  DB  support  increases  N  plaborm  test   –  Large  scale  tests  are  serialized  due  to  resources   © 2013 VMware, Inc. All rights reserved
  9. 9. Plan:  Embedded  Default  DB   •  Embed  PostgreSQL  as  part  of  the  product   •  Make  it  the  default  choice  opXon  for  most  deployment   •  Do  not  expose  database  directly  to  endusers   •  ApplicaXon  will  manage  the  DB   •  Support  it  for  producXon  deployment   •  Customer  can  sXll  select  to  use  an  external  database  of   their  choice  or  save  DB  License  cost  with  the   embedded  one     © 2013 VMware, Inc. All rights reserved  
  10. 10. Ground  Work:  MigraXon   •  Products  used  various  databases   –  SQL  Server   –  Oracle     –  MySQL   •  Few  had  about  1500  stored  procedures   •  Some  used  exoXc  features   •  For  example    MySQL  used  InMemory  Engine,  InnodB   engine,  MyISAM  engine  in  the  same  workload   © 2013 VMware, Inc. All rights reserved
  11. 11. MigraXon   •  Leveraged  contractors  to  migrate  stored   procedures   •  Performance  bake-­‐off  tests  to  convince   product  teams   •  Integrated  into  the  build  infrastructure  to   ease  pulling  components   © 2013 VMware, Inc. All rights reserved
  12. 12. Example:  vCenter  Server  Appliance   © 2013 VMware, Inc. All rights reserved
  13. 13. Plan:  Backup  &  Recovery   •  DB  not  the  only  thing  to  be  backed  up   •  ConfiguraXon  files   •  ApplicaXon  Status   •  All  data  to  be  backed  up  went  into  a  virtual  disk  backed   up  with  vADP   •  pg_dump  /  pg_restore  used  in  few  cases     © 2013 VMware, Inc. All rights reserved
  14. 14. IniXal  Embedded  DB  Deployments   •  Each  Product  team  is  different   –   has  different  levels  of  DB  Development  skills   •  Each  architect  comes  up  with  their  own  design     –  Postgres  Service  designed  (Start/Stop)   –  own  DB  Layout  design   –  own  DB  Security  design   –  Some  Tuned  their  DB   –  Many  did  not   © 2013 VMware, Inc. All rights reserved
  15. 15. Early  Results   •  YES  !  Customer  started  seeing  benefit   •  However   –  Experience  of  PostgreSQL  different  on  each  product   –  Random  security  holes  idenXfied  due  to  lack  of   knowledge  of  PostgreSQL  nuances   –  Performance  configuraXon  was  an  issue   –  Upgrade  to  new  versions  of  PostgreSQL  was  an   akerthought  (What  no  in-­‐place  upgrade?)   –  Legal  work  required  for  OSS  increased   –  Resulted  in  PostgreSQL  version  sXckiness   © 2013 VMware, Inc. All rights reserved
  16. 16. Some  encounters   •  Portable  binaries  –  Shrink  wrapped  sokware   (not  compiled  on  same  systems)   –  OSS  Libraries  used  for  compiling  PostgreSQL  have   to  be  shipped     –  Manage  security  patches  for  OSS  Libraries   •  Need  Standard  drivers  like  ODBC,  JDBC   © 2013 VMware, Inc. All rights reserved
  17. 17. MulX-­‐Plaborm  Requirement   •  As  company  grew,  acquisiXons  grew   •  Some  products  only  worked  on  Windows   •  Few  used  exclusively  .NET  framework   © 2013 VMware, Inc. All rights reserved
  18. 18. SoluXon:  Embedded  DBaaS   •  Provide  End  to  End  Experience  with   PostgreSQL   © 2013 VMware, Inc. All rights reserved
  19. 19. InstallaXon   •  Extend  installaXon  to  setup  a  service   •  Configurable  to  set  up  as  internal  service  or   external  service  (mostly  embedded)   •  Security  handled  as  part  of  service  setup  to   follow  best  pracXcesfor  embedded   •  Security  based  on  access  and  minimum  roles   defined   © 2013 VMware, Inc. All rights reserved
  20. 20. InstallaXon   •  RPM    installaXon   –  Dependent  OSS  Libraries  to  make  RPM  portable   –  Server  and  client  RPMs   –  Service  Setup  configurator  to  setup  a  default  service   •  •  •  •  User  setup   initdb  setup   pg_hba  setup     Password  handling   –  Provide  tools  to  quickly  setup  DB  Owner,  DB  for   various  users  of  the  service   © 2013 VMware, Inc. All rights reserved
  21. 21. Installer  on  Windows   •  •  •  •  PostgreSQL  Installer  on  Windows   Service  setup  on  Windows   ODBC  Support     Npgsql  Support  for  .NET  Plaborms   © 2013 VMware, Inc. All rights reserved
  22. 22. Backup  &  Recovery   •  While  other  backups  strategy  are  available   –  Nothing  beats  standard  database  backups   –  Design  cron  jobs  for  criXcal  data     –  Design  Archive  On  Demand  for  semng  up   replicaXon  quickly   –  Most  customer  do  not  want  to  hack  with  backup/ restore  scripts.     –  Should  work  out  of  the  box   © 2013 VMware, Inc. All rights reserved
  23. 23. Upgrades   •  Seamless  upgrade  to  new  version   –  RPM  Upgrades  needs  to  be  transparent  for  minor   versions   –  RPM  upgrades  for  major  version  needs  to  migrate   data  from  the  older  version  in  the  default  service   © 2013 VMware, Inc. All rights reserved
  24. 24. Upgrades   •  Problems  sXll  not  completely  resolved   –  Reduce  downXme  for  major  release  upgrades   –  Generally  migraXon  since  in-­‐place  upgrade  does   not  provide  ways  to  roll  back  the  upgrade   © 2013 VMware, Inc. All rights reserved
  25. 25. Performance   •  Default  PostgreSQL  semng  are  conservaXve   •  Most  deployments  use  default  values   •  Started  doing  auto-­‐tuning   –  Modified  PostgreSQL  core  engine  with  intelligence   –  For  checkpoint  segments,    shared_memory   •  Added  tool  to  auto  tune  postgresql.conf     –  Based  on  memory  share  for  DB  Service   –  Product  workload  dependent     © 2013 VMware, Inc. All rights reserved
  26. 26. Performance  related  tools   •  Provided  tools  to  understand  what’s   happening  with  Postgres   –  pg_top  with  changes   •  Changes  already  contributed  back  to  community   © 2013 VMware, Inc. All rights reserved
  27. 27. Example:  pg_top   © 2013 VMware, Inc. All rights reserved
  28. 28. Performance  related  future  work   •  ExisXng  tools  sXll  meant  for  expert  users   •  Lot  of  demand  for  simple  performance  reports   •  Looking  into  pg_statspack   © 2013 VMware, Inc. All rights reserved
  29. 29. Availability   •  Created  a  watchdog  for  PostgreSQL  to  restart   PostgreSQL  if  database  cannot  be  accessed  for   long   •  If  restart  fails  then  it  can  throw  alerts   •  Also  created  plugins  for  monitoring  the   service   © 2013 VMware, Inc. All rights reserved
  30. 30. Related  Problems:  Crashes   •  PostgreSQL  started  throwing  errors  with  “Out   of  Memory”  errors  in  stress  tests     •  MulXple  applicaXons  can  fight  for  memory   •  Made  Performance  semngs  less  aggressively   © 2013 VMware, Inc. All rights reserved
  31. 31. Supportability   •  When  customers  hit  a  problem   –  Hard  to  get  access  to  the  system   •  Created  support  package  tool  to  collect   relevant  informaXon   –  Collect  config,  status  in  a  tar  file  for  analysis   © 2013 VMware, Inc. All rights reserved
  32. 32. Usability   •  Experts  love  command  line  uXliXes   •  Novices  love  GUI,  WebUI   •  It  needs  to  match  the  other  GUIs  VMware  ship   © 2013 VMware, Inc. All rights reserved
  33. 33. Pluggable  UI   •  Matching  VMware’s  GUI  Plaborm   © 2013 VMware, Inc. All rights reserved
  34. 34. Embedded  DBaaS  Architecture   VMware   Product     Service       Configurator     WebUI       Pg_top         JDBC     PostgreSQL     ODBC       Linux  Based  VM   Watchdog   Perf-­‐Tuner   Monitoring   Support   package  tool   vSphere  Plaborm   © 2013 VMware, Inc. All rights reserved
  35. 35. Virtual  Appliances   © 2013 VMware, Inc. All rights reserved
  36. 36. Virtual  Appliances   © 2013 VMware, Inc. All rights reserved
  37. 37. ReplicaXon  between  vApps   •  Provide  simple  uXliXes  to  quickly   –  archive_command   –  create_replicaXon_user   –  run_as_replica   –  show_replica_status   –  promote_replica_to_primary   © 2013 VMware, Inc. All rights reserved
  38. 38. In  Closing   •  Free  Sokware  is  nice   •  Experience  –  What  business  users  care   –  User  Experience   –  DBA  Experience   –  Support  Experience   –  etc   •  Improve  the  Experience  with  the  technology   © 2013 VMware, Inc. All rights reserved
  39. 39. Thanks.   QuesXons?   Download  VMware  vFabric  Postgres     hKp://vmware.com/go/download-­‐vfabric-­‐postgres   Follow us on twitter: @vPostgres Data Platform Blog: http://blogs.vmware.com/dataplatforms © 2013 VMware, Inc. All rights reserved
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×