Successfully reported this slideshow.

BETA: "Cloud Developers DHARMA: Redefining 'done' for Cloud applications"

423 views

Published on

This is a beta version of a talk I hope to present at a few Java / Software Development conferences this year. Comments are most welcome!

Published in: Technology
  • Be the first to comment

BETA: "Cloud Developers DHARMA: Redefining 'done' for Cloud applications"

  1. 1. Cloud  Developer’s  DHARMA…    redefining  ‘done’  for  Cloud  applica3ons   Daniel  Bryant   CTO,  Instant  Access  Technologies     d.bryant@iatltd.com   @taidevcouk    
  2. 2. 19/02/2014   @taidevcouk  
  3. 3. epoints.com  2012/13  Upgrade…   •  Increasing  traffic     –  Scalability  being  stretched   •  Increasingly  diverse  requirements   •  Our  starPng  point   –  Developers  creaPng  monolithic  applicaPon   –  Manual  QA   –  OperaPons  deploying  to  data  center   19/02/2014   @taidevcouk  
  4. 4. Core  Changes…   •  Service-­‐Oriented  Architecture   •  Cloud-­‐based  deployments   •  DevOps  Culture   19/02/2014   @taidevcouk  
  5. 5. Core  Changes…   •  Service-­‐Oriented  Architecture   –  TwiTer’s  Story  (bit.ly/1j1WbmI)   •  Cloud-­‐based  deployments   –  Tonight!   •  DevOps  Culture   –  LJC  Event  March  13th  7pm   19/02/2014   @taidevcouk  
  6. 6. Moving  to  the  Cloud   •  IAT  chose  Amazon  Web  Services  (AWS)  IaaS   •  Great  choice,  but…   –  Made  a  few  mistakes     –  Learnt  lots  of  lessons   19/02/2014   @taidevcouk  
  7. 7.     We’ve  created  a  “Cloud  Developer’s  DHARMA”   to  act  as  a  checklist  when  building  Cloud  apps   19/02/2014   @taidevcouk  
  8. 8. dharma   /ˈdɑːmə,ˈdəːmə/     noun     1.  Signifies  behaviors  that  are  considered  to  be  in   accord  with  order  that  makes  life  and  universe   possible  (Hinduism)   2.  "cosmic  law  and  order”,  but  is  also  applied  to   the  teachings  of  the  Buddha  (Buddhism)   19/02/2014   @taidevcouk  
  9. 9. Documented  (at  the  edge)   Highly  cohesive/loosely  coupled  (all  the  way  down)   Automated  from  commit  to  Cloud   Resource  aware   Monitored  thoroughly   AnPfragile   19/02/2014   @taidevcouk  
  10. 10. Documented  (at  the  edge)   Highly  cohesive/loosely  coupled  (all  the  way  down)   Automated  from  commit  to  Cloud   Resource  aware   Monitored  thoroughly   AnPfragile   19/02/2014   @taidevcouk  
  11. 11. 19/02/2014   @taidevcouk  
  12. 12. Edge  DocumentaPon  is  Vital   •  Swagger   –  helloreverb.com/developers/swagger   •  Mashery’s  IODocs   –  www.mashery.com/product/io-­‐docs   •  Google’s  Discovery  Documents   –  developers.google.com/discovery/v1/using   19/02/2014   @taidevcouk  
  13. 13. 19/02/2014   @taidevcouk  
  14. 14. 19/02/2014   @taidevcouk  
  15. 15. 19/02/2014   @taidevcouk  
  16. 16. Documented  (at  the  edge)   Highly  cohesive/loosely  coupled  (all  the  way  down)   Automated  from  commit  to  Cloud   Resource  aware   Monitored  thoroughly   AnPfragile   19/02/2014   @taidevcouk  
  17. 17. High  Cohesion  /  Loose  Coupling…   …all  the  way  down     •  Public  API   –  PayPal  (bit.ly/1hnZNly)   •  Architecture   –  Services   –  Components   19/02/2014   @taidevcouk  
  18. 18. High  Cohesion  /  Loose  Coupling…   •  Code   –  SOLID  Principles   •  Datastores  /  Caches   –  Fault  tolerance   –  Zero-­‐downPme  deployments   –  MigraPon  and  upgrades   19/02/2014   @taidevcouk  
  19. 19. Documented  (at  the  edge)   Highly  cohesive/loosely  coupled  (all  the  way  down)   Automated  from  commit  to  Cloud   Resource  aware   Monitored  thoroughly   AnPfragile   19/02/2014   @taidevcouk  
  20. 20. Automated  from  Commit  to  Cloud   •  ConPnuous  IntegraPon/Delivery   •  Jenkins  Build  Pipeline  Plugin  (bit.ly/1hq9XSN)   19/02/2014   @taidevcouk  
  21. 21. Our  Build  Pipeline   •  Component  Build     –  Compile   –  Unit  Tests  (surefire)   –  IntegraPon  Tests  (failsafe)   •  Deployment  onto  QA  Cloud   –  Python  Scripts  +  Chef  to  provision   –  Verify  success  using  Python   19/02/2014   @taidevcouk  
  22. 22. Our  Build  Pipeline   •  Acceptance  Tests   –  Cucumber  and  Selenium   –  Work  in  progress…   •  Performance  Tests   –  Jmeter   –  Jenkins  Jmeter  performance  plugin   •  Staging  /  Live  Deployment   –  Human-­‐based  condiPonal  operaPon     19/02/2014   @taidevcouk  
  23. 23. Documented  (at  the  edge)   Highly  cohesive/loosely  coupled  (all  the  way  down)   Automated  from  commit  to  Cloud   Resource  aware   Monitored  thoroughly   AnPfragile   19/02/2014   @taidevcouk  
  24. 24. Deployment  Planorm:  What  you’ve  got…   19/02/2014   @taidevcouk  
  25. 25. What  you  think  you  want…   19/02/2014   @taidevcouk  
  26. 26. What  you  get…   Fact:  9  out  of  10  cheetahs  prefer  the  taste  of  an  Ops  team  over  Enned  food     19/02/2014   @taidevcouk  
  27. 27. Thou  Shalt  Know  thy  Cloud…   •  AWS  EBS  100  IOPS  (by  default)     –  My  Mac  SSD  does  49K  IOPS   •  1000Mbps  network  max  transfer  ~125Mb/s   –  My  Mac  does  400+  Mb/s  SequenPal  Write  to  SSD    Reference  for  Mac  staPsPcs:  bit.ly/1tJZH8   19/02/2014   @taidevcouk  
  28. 28. Thou  Shalt  Know  thy  Cloud…   •  “Noisy  [virtual]  Neighbours”   –  My  Mac  only  deals  with  noisy  colleagues   •  “Things  fail  all  the  Pme  [in  the  Cloud]”   –  My  Mac  is  quite  reliable…     19/02/2014   @taidevcouk  
  29. 29. Documented  (at  the  edge)   Highly  cohesive/loosely  coupled  (all  the  way  down)   Automated  from  commit  to  Cloud   Resource  aware   Monitored  thoroughly   AnPfragile   19/02/2014   @taidevcouk  
  30. 30. Monitor  All  The  Things!   •  Infrastructure  monitoring   –  Nagios   –  Zabbix   –  Splunk   –  AppDynamics   19/02/2014   @taidevcouk  
  31. 31. Component  Metrics   •  Dropwizard’s  Metrics   –  metrics.codahale.com   •  Nenlix’s  Servo   –  github.com/Nenlix/servo   •  Etsy’s  StatsD   –  github.com/etsy/statsd/wiki   19/02/2014   @taidevcouk  
  32. 32. Gauges,  Counters,  Meters,  Timers…   19/02/2014   @taidevcouk  
  33. 33. Health  Checks   19/02/2014   @taidevcouk  
  34. 34. Documented  (at  the  edge)   Highly  cohesive/loosely  coupled  (all  the  way  down)   Automated  from  commit  to  Cloud   Resource  aware   Monitored  thoroughly   AnPfragile   19/02/2014   @taidevcouk  
  35. 35. AnPfragile   19/02/2014   @taidevcouk  
  36. 36. Black  Swan  Theory     “…is  a  metaphor  that  describes  an  event  that   comes  as  a  surprise,  has  a  major  effect,  and  is   oten  inappropriately  raPonalized  ater  the  fact   with  the  benefit  of  hindsight”       Nassim  Nicholas  Taleb,  The  Black  Swan   19/02/2014   @taidevcouk  
  37. 37. AnPfragile   •  The  opposite  of  fragile?   –  Robust…   –  AnPfragile…   •  Nenlix  are  best-­‐in-­‐class   –  bit.ly/1gs5n3q   •  System  must  be  robust  first!   19/02/2014   @taidevcouk  
  38. 38. Design  for  Failure   •  Design  paTerns   –  Timeouts  /  retries   –  Bulkheads  /  circuit-­‐breakers   •  InspiraPon   –  Chris  Richardson  (slidesha.re/1t3vsg)   –  Nenlix  (bit.ly/1h5GMid)   19/02/2014   @taidevcouk  
  39. 39. Real  AnPfragility   •  Autoscaling   19/02/2014   @taidevcouk  
  40. 40. AnPfragile  PaTerns   •  Stateless  components   •  Distributed  data  stores  /  caches   •  Eventual  consistency   •  Asynchronous  communicaPon   –  Message  /  event  driven   19/02/2014   @taidevcouk  
  41. 41. So,  Cloud  Apps  are  ‘done’  when…   Documented  (at  the  edge)   Highly  cohesive/loosely  coupled  (all  the  way  down)   Automated  from  commit  to  Cloud   Resource  aware   Monitored  thoroughly   AnPfragile   19/02/2014   @taidevcouk  
  42. 42. Thanks  For  Listening   •  Massive  thanks  to  all  the  IAT  team!   •  This  is  a  beta  version  talk     –  Feedback  is  most  appreciated!   •  QuesPons  /  comments?   –  d.bryant@iatltd.com   –  @taidevcouk   19/02/2014   @taidevcouk  

×