Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous Delivery @ Onegini

This presentation shows how we at Onegini do Continuous Delivery. It takes you through the challenges we encountered and the choices we made.

  • Be the first to comment

  • Be the first to like this

Continuous Delivery @ Onegini

  1. 1. Continuous Delivery How it’s done at a SAAS start-up Presenter: Stein Welberg
  2. 2. About Me Studied at Twente University Waterpolo Player Identity & Access management expert CTO @ Onegini
  3. 3. Why am I here? Sell Onegini Explain about Continuous Delivery Inspire you to think about CD Steal engineers ;-)
  4. 4. What is Onegini SaaS Platform: SSO solution to give external users access to your organisation Focus on ease of use Focus on security
  5. 5. Continuous Integration "Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly." - Martin Fowler
  6. 6. Continuous Delivery Key concepts Release early, release often Small releases Development close to business Test the sh*t out of your application Not just the code, also infra setup, db schema, etc! Have software that can always go live! Treat anything as software Automate, automate, automate… automate!!! Full deployment pipeline Summary: Continuous Delivery is Continuous Integration on steriods
  7. 7. Continuous Delivery @ Onegini A new release into production every 4 weeks Devs could release anytime, business can’t keep up with us ;-) Fully automated Deployments (infrastructure as code) For all Business critical applications Automated (integration / acceptance) testing Critical issues are hardly ever found by our testers Not all corner cases are tested automatically
  8. 8. Our road to Continuous Delivery The challenges Ruby (on Rails) and Java combined. Could not easily drop a war into e.g. Google App engine or Azure. Plans to use a graph / nosql database in the near future Run (and be able to customize) and roll own virtual machine Completely automated build environment testable and repeatable No (manual) installation or installation manual ;-) No SSH to the servers No manual database changes Time needed to update the production cluster must be less than one hour
  9. 9. Choices we made Cloud Provider: Amazon Advantages IAAS: so you can host anything you want We didn’t know exactly how many machines we needed or what kind of software stack was necessary to complete the job A lot of tooling exists Great API’s Relatively cheap when you are small Free tier Disadvantages Small machines are really slow Speed to cost ratio is quite bad
  10. 10. Choices we made Rolling entire machines Advantages Machines stay fresh Every machine is exactly the same (does not matter which environment) Up- and downscaling is very easy and can be automated Creating a machine can be automated and tested Disadvantages Need to manually run OS updates if you don’t roll machines often enough No state inside the machine Database needs to be backwards compatible Property files need to be present on the machines (can be extracted)
  11. 11. Choices we made AWS RDS Database Advantages MySQL or other DB type (Oracle, MS SQL) is available Easy to setup Disadvantages Only marginal customisability. Heavy performance tuning is not possible. Write performance is not great Backup and restore results in a different hostname
  12. 12. Choices we made Installation automation: Chef Solo Advantages Cookbooks in ruby Standalone No need to manually login to a server Disadvantages Updating cookbooks does break our build sometimes Dependant of the community for updates of cookbooks We have forked some cookbooks because they were broken or out of date Troubleshooting can be time consuming
  13. 13. Choices we made Base image Advantages Saves time: not necessary to install / run initial OS updates on the clones Install software that is necessary on the entire stack Disadvantages If the base AMI contains an error all images do! Base AMI based on Application Application AMI's AMI's uses Application Instance
  14. 14. Choices we made Asgard Open-source tool by Netflix written in Groovy Web-based AWS cloud management and deployment Advantages It provides the “rolling entire machines” approach Uses the Amazon API’s Provides it’s own API’s for automation Amazon does not provide a UI to create Auto scaling groups Disadvantages Can only be used on Amazon AWS
  15. 15. Asgard Cloud deployment model
  16. 16. Asgard Cloud deployment model App 1 App 2 App n
  17. 17. Asgard Fast rollback
  18. 18. Asgard Fast rollback
  19. 19. Asgard Fast rollback
  20. 20. Asgard Fast rollback
  21. 21. Asgard Fast rollback
  22. 22. Asgard Fast rollback
  23. 23. Build Architecture Overview local Dev commit test trigger Build pipeline Jenkins CI + detect create Asgard test create use + Subversion AMI Inst ances
  24. 24. Demo Asgard Chef Cookbook Deploying a new version to test
  25. 25. Lessons learned Needed to clean up old Amazon AMI’s, volumes, launch configurations, etc A lot of custom ruby scripts AMI builder Base AMI builder Cleanup Custom way to prevent building the entire pipeline in jenkins Automating to starting a new machine and connecting to it via SSH is quite ugly.. Amazon reuses IP addresses, this sometimes results in a fingerprint conflict which causes a pipeline failure
  26. 26. The future Amazon VPC setup Network segmentation In and egress filtering Migration to another data center Host data in a data center that is not owned by an American organisation