Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Owain Perry (Just Giving) - Continuous Delivery of Windows Micro-Services in the Cloud

Owain will talk about the journey JustGiving.com have gone through to get to Continuous delivery on their Windows environment. He will talk about what they did, how they did it and lessons learned along the way

Video: https://www.youtube.com/watch?v=MVXaR6oEK60

Join DevOps Exchange London here: http://www.meetup.com/DevOps-Exchange-London
Follow DOXLON on twitter http://www.twitter.com/doxlon

  • Be the first to comment

Owain Perry (Just Giving) - Continuous Delivery of Windows Micro-Services in the Cloud

  1. 1. Continuous Delivery of Windows microservices in the cloud @owainperry http://www.slideshare.net/perryofpeek/continuous-delivery-of-windows-micro-services-in-the-cloud
  2. 2. What? ● The journey of the last 18 months, Justgiving has taken to ship more in less time ● How we have implemented supporting infrastructure around micro services ● Where we went wrong
  3. 3. ● Raise Money for Charities ● Turnover ~£400 Million a year ● Process UK gift aid ● Text Giving ● Crowd funding ● ~1000 nodes
  4. 4. Stephen Sutton
  5. 5. Monolith legacy ● High developer pain ● Branching ● 2 week release cycles ● Long build times ● Long test times ● Massive git repository ● Legacy source code patterns (bin files) ● etc…...
  6. 6. Micro services
  7. 7. JustGiving micro service? ● It's a small service! – Usually a component makes up part of a business domain – Has one purpose – Is simple to reason about – Reality - various levels of granularity
  8. 8. Where we went next ● New stuff: – built using micro services – Hosted in the cloud – New pipelines for deployment ● Monolith stuff: – Chop out choice areas, bite size chunks at a time – Build a simpler pipeline – Simpler branching strategy
  9. 9. Infrastructure principles ● “Very” Available ● Repeatable through automation ● Development define and build their infrastructure ● Keep environments isolated ● Be pragmatic….
  10. 10. Train track automation ● Development - self service infrastructure – Fast to spin up infrastructure – Less Ops bottleneck ● Reduce the need for AWS keys ● Consistency benefits: – Service architecture – Names and name conventions – Tags – Security groups – Scripting
  11. 11. Types of service ● 1. Windows service Either: – Self hosted nancyfx web server ● Application ● Health endpoint – Message based service ● Health endpoint ● 2. IIS hosted application – Health endpoint
  12. 12. Core tools ● AWS ● Artifactory – package repository ● Chef – open source v11 ● GO CD ● Team City ● Chocolatey
  13. 13. Auto scale groupAuto scale group Micro service hosting architecture ● Varnish is the load balancer Elastic Load BalancerElastic Load Balancer Varnish Node 1 Node 2 Node 1Node 2
  14. 14. CD of micro services Each micro service has its own ● Git repository ● GO Pipeline ● Team City build ● Chef cookbook ● Tests ● AWS Infrastructure ● Operations run book ● Health check endpoint - /status/health ● Blue / Green Deployment
  15. 15. Steps for a new micro service ● Think of a name ● Create a git repository ● Open up micro service template in visual studio – Rename a few things ● Teamcity build (copy n paste) ● Go pipeline (copy n paste) ● Define Infrastructure via json
  16. 16. The pipeline Teamcity Git Chef (cookbook) Go CD Artifactory (choco pkg) Dev Staging Prod Sandbox
  17. 17. Versions ● Application chocolatey package ● Cookbook
  18. 18. AWS infrastructure ● Treat AWS as another resource ● Heavy Weight Resource Providers for AWS ● Light weight resource Providers to abstract the HWRP ● json data_bags based infrastructure ● Infrastructure builder node executes chef client for infrastructure ● “Most” of our infrastructure is built this way, not just business application services
  19. 19. LWRP – data bag - json { "id": "dev", "type": "asg-lcf", "region": "eu-west-1", "availability_zones": ["eu-west-1a" ], "vpc": "DEV-VPC", "routetable": "RTB-DEV-PRIVATE", "ami": "JG-UBUNTU12-04-V5", "instance_type": "m3.medium", "security_groups": ["********" ], "iam_role": "instance_default", "launch_configuration_name": "LCF-DEV-SENSU", "aws_autoscale_group_name": "ASG-DEV-SENSU", "health_check_type": "EC2", "health_check_grace_period": 300, "asg_min_size": 3, "asg_max_size": 3, "asg_desired_capacity": 3, "asg_default_cooldown": 200, "subnets": [ { "name": "SUB-DEV-MONITORING-ZA",
  20. 20. Micro service GO pipeline
  21. 21. Micro service cookbook ● Chef is used to configure the node ● Chef is used to deploy the application ● “deployment” - is just version constraint in a chef environment
  22. 22. Deployment ● Set the version constraint for the inactive environment ● Chef search all nodes ● For each node { remote in an kick off chef client scheduled task} ● Switch active environment ● Poll for nodes to be on correct version. ● Post deployment tests
  23. 23. Consistency in architecture ● Name conventions ● Build scripts ● Packages ● Supports ● Monitoring ● Refactoring ● Comprehension
  24. 24. Bootup ● Chef is prebaked into AMI (faster) ● Instance userdata script contains the bootup script (inc runlist) ● Auto scale groups used to instantiate new instances
  25. 25. Automated change ● Change log automated into Jira for releases – Staging and production ● Change log automated for cookbook releases ● Platform to include more over time.
  26. 26. Automated change log
  27. 27. Information
  28. 28. What we didn't get right... – Zombies…. – Chocolatey (exe / bat) – AWS reboot – Cost control – Abstraction of infrastructure json – Chef search to configure load balancer – Not automating our base AMI's from the outset
  29. 29. Future ● Rolling instance recycling – daily / weekly? ● Bake images in production ● Ohai for sensu subscriptions ● Service discovery rather than chef search ● Application secret security ● Code security scanning in the pipelines ● Infrastructure penetration testing ● Infrastructure security checks ● Packer to build base ami's
  30. 30. So…. – Micro services – has helped productivity – AWS automation in hands of developers – Simple json defined infrastructure – Chef is core at the centre – We have made mistakes – But it's been a worthwhile journey so far.
  31. 31. Questions? @owainperry http://www.slideshare.net/perryofpeek/continuous-delivery-of-windows-micro-services-in-the-cloud
  32. 32. Additional
  33. 33. Chef resources ● Elastic Load Balancer ● Auto scale groups ● Security groups ● IAM roles ● Launch configurations ● DNS ● Subnets ● Ec2 instances ● Route tables
  34. 34. HWRP example aws_subnet 'subtest2' do accesskey accesskey secretkey secretkey region “eu-west-1” vpc “DEV-VPC” azone 'eu-west-1a' cidr '10.20.30.0/24' routetable 'RTB-STG-PRIVATE' tags [ { "a" => "1"},{"c" => "2" }] action :create end
  35. 35. Pipelines for chef cookbooks
  36. 36. Chocolatey ● Rebuild all packages for internal hosting. – Don't want a dependency on 3rd party download – Control versions of software in use ● Common pattern for git and builds – Take ~10 mins to repackge ● ( This is a bit of a chore )

×