Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
THE ROAD TO CONTINUOUS DEPLOYMENT - A CASE STUDY MICHIEL ROOK - @MICHIELTCS
▸ Java, PHP & Scala developer ▸ Consultant, trainer, speaker ▸ make.io ▸ Dutch Web Alliance ▸ Maintainer of Phing ▸ @michi...
THIS TALK ▸ Background ▸ The approach ▸ Process / standards ▸ Build pipelines ▸ Results & lessons learned
THE SYSTEM - SAN DIEGO ▸ ... or the Big Ball Of Mud ▸ Large legacy monolith ▸ Generates significant income ▸ Slow & complex...
SAN DIEGO FRONTEND MYSQL DB SAN DIEGO BACKEND LOAD BALANCERS / VARNISH ITBANEN INTERMEDIAIR NATIONALEVACATUREBANK SAN DIEG...
THE SYSTEM - SAN DIEGO ▸ Infrequent, manual releases ▸ Fragile tests ▸ Low velocity ▸ Frequent outages / bugs / issues ▸ F...
GOALS ▸ Reduce issues ▸ Reduce cycle time ▸ Increase productivity ▸ Increase motivation
REFACTOR? REBUILD?
APPROACH ▸ Strangler pattern ▸ Services per domain object (job, jobseeker, ...) ▸ Proxy to switch between old/new ▸ Migrat...
ORIGINAL MONOLITH PROXY SERVICE ORIGINAL MONOLITH ORIGINAL MONOLITH SERVICE SERVICE SERVICE PROXY DB DB DB DB DB DB
APPROACH ▸ Services behind load balancers ▸ Access legacy db’s ▸ Continuous deployment ▸ Docker containers ▸ Frontends are...
SAN DIEGO ELASTIC SEARCHLEGACY DB JOB SERVICE RMQ ITBANEN INTERMEDIAIR NATIONALEVACATUREBANK MONGO DB ITBANEN JOBSEEKER SE...
PROCESS
STARTING OFF ▸ Scrum, 1 week sprints ▸ TDD / BDD ▸ Definition of Done ▸ Team mindset / experience ▸ Focus on value ▸ Replac...
CONTINUOUS EVERYTHING
DEV BUILD / TEST CONTINUOUS INTEGRATION
DEV BUILD / TEST ACCEPTANCE PRODUCTION CONTINUOUS DELIVERY
DEV BUILD / TEST ACCEPTANCE PRODUCTION CONTINUOUS DEPLOYMENT
WHY CONTINUOUS DEPLOYMENT ▸ Small steps ▸ Less overhead ▸ Early feedback ▸ Reduce cycle time ▸ Reduce risk
ONLY COMMIT TO MASTER
NO BRANCHES
NO BRANCHES REALLY.
PAIR PROGRAMMING
BOY SCOUT RULE
QUALITY GATES
100% CODE COVERAGE*
FEATURE TOGGLES, A/B TESTS
DEVOPS
DASHBOARDS
BUILD
 PIPELINE
AUTOMATE REPEATABLE THINGS
EVERY COMMIT GOES TO PRODUCTION
DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTS
DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE UI TESTS public function testJobCannotBeFound() {
 $jobRepositor...
DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTS public function testFindJob() {
 $expectedJob = ...
DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTSScenario: Link to related job
 Given a job exists...
DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTS
DEFENSE IN DEPTH MANUAL TESTING? UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI
CONTINUOUS TESTING UNIT TESTS INTEGRATION TESTS ACCEPTANCE TESTS UI TESTS SMOKE
 TESTS Cost Speed Exploratory
 testing Mon...
PIPELINE AS CODE node {
 stage 'Run tests'
 sh "phpunit"
 sh "behat"
 
 stage 'Build docker image'
 sh "phing build"
 sh "...
DOCKERFILE FROM php:7.0-apache ADD vhost.conf /etc/apache2/sites-available/000-default.conf ADD . /var/www/html
DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONT...
DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONT...
DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONT...
DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONT...
DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONT...
DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONT...
DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONT...
DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONT...
BUILD PIPELINE
FEEDBACK!
RESULTS
RESULTS ▸ Total build time per service < 10 minutes ▸ Significantly improved page load times ▸ Improved audience stats (tim...
LESSONS LEARNED ▸ Team acceptance ▸ Change is hard ▸ Overhead of weekly sprint; requires discipline ▸ Docker orchestration...
LESSONS LEARNED ▸ Experience with new tech ▸ Stability of build pipelines ▸ Management/leadership buy-in ▸ Business alignm...
QUESTION TIME
THANK YOU! @michieltcs / michiel@make.io
The road to continuous deployment (PHPCon Poland 2016)
Upcoming SlideShare
Loading in …5
×

The road to continuous deployment (PHPCon Poland 2016)

2,337 views

Published on

As presented at PHPCon Poland 2016

It's a situation many of us are familiar with: a large legacy application, limited or no tests, slow & manual release process, low velocity, no confidence.... Oh, and management wants new features, fast.
But how to proceed? Using examples and lessons learned from a real-world case, I'll show you how to strangle the legacy application with a modern service architecture and build a continuous deployment pipeline to deliver value from the first sprint. On the way, we take a look at testing strategies and various (possibly controversial!) tips and best practices.

Published in: Software
no profile picture user

  • Login to see the comments

The road to continuous deployment (PHPCon Poland 2016)

  1. 1. THE ROAD TO CONTINUOUS DEPLOYMENT - A CASE STUDY MICHIEL ROOK - @MICHIELTCS
  2. 2. ▸ Java, PHP & Scala developer ▸ Consultant, trainer, speaker ▸ make.io ▸ Dutch Web Alliance ▸ Maintainer of Phing ▸ @michieltcs
  3. 3. THIS TALK ▸ Background ▸ The approach ▸ Process / standards ▸ Build pipelines ▸ Results & lessons learned
  4. 4. THE SYSTEM - SAN DIEGO ▸ ... or the Big Ball Of Mud ▸ Large legacy monolith ▸ Generates significant income ▸ Slow & complex ▸ Technical debt
  5. 5. SAN DIEGO FRONTEND MYSQL DB SAN DIEGO BACKEND LOAD BALANCERS / VARNISH ITBANEN INTERMEDIAIR NATIONALEVACATUREBANK SAN DIEGO FRONTEND SAN DIEGO FRONTEND SAN DIEGO FRONTEND SAN DIEGO BACKEND SAN DIEGO BACKEND SAN DIEGO BACKEND MEMCACHE FTP EXT. SERVICES SOLR
  6. 6. THE SYSTEM - SAN DIEGO ▸ Infrequent, manual releases ▸ Fragile tests ▸ Low velocity ▸ Frequent outages / bugs / issues ▸ Frustrated team ▸ Low confidence modifying existing code
  7. 7. GOALS ▸ Reduce issues ▸ Reduce cycle time ▸ Increase productivity ▸ Increase motivation
  8. 8. REFACTOR? REBUILD?
  9. 9. APPROACH ▸ Strangler pattern ▸ Services per domain object (job, jobseeker, ...) ▸ Proxy to switch between old/new ▸ Migrate individual pages
  10. 10. ORIGINAL MONOLITH PROXY SERVICE ORIGINAL MONOLITH ORIGINAL MONOLITH SERVICE SERVICE SERVICE PROXY DB DB DB DB DB DB
  11. 11. APPROACH ▸ Services behind load balancers ▸ Access legacy db’s ▸ Continuous deployment ▸ Docker containers ▸ Frontends are services
  12. 12. SAN DIEGO ELASTIC SEARCHLEGACY DB JOB SERVICE RMQ ITBANEN INTERMEDIAIR NATIONALEVACATUREBANK MONGO DB ITBANEN JOBSEEKER SERVICE NVBINTERMEDIAIR
  13. 13. PROCESS
  14. 14. STARTING OFF ▸ Scrum, 1 week sprints ▸ TDD / BDD ▸ Definition of Done ▸ Team mindset / experience ▸ Focus on value ▸ Replace old features with new (legacy becomes obsolete)
  15. 15. CONTINUOUS EVERYTHING
  16. 16. DEV BUILD / TEST CONTINUOUS INTEGRATION
  17. 17. DEV BUILD / TEST ACCEPTANCE PRODUCTION CONTINUOUS DELIVERY
  18. 18. DEV BUILD / TEST ACCEPTANCE PRODUCTION CONTINUOUS DEPLOYMENT
  19. 19. WHY CONTINUOUS DEPLOYMENT ▸ Small steps ▸ Less overhead ▸ Early feedback ▸ Reduce cycle time ▸ Reduce risk
  20. 20. ONLY COMMIT TO MASTER
  21. 21. NO BRANCHES
  22. 22. NO BRANCHES REALLY.
  23. 23. PAIR PROGRAMMING
  24. 24. BOY SCOUT RULE
  25. 25. QUALITY GATES
  26. 26. 100% CODE COVERAGE*
  27. 27. FEATURE TOGGLES, A/B TESTS
  28. 28. DEVOPS
  29. 29. DASHBOARDS
  30. 30. BUILD
 PIPELINE
  31. 31. AUTOMATE REPEATABLE THINGS
  32. 32. EVERY COMMIT GOES TO PRODUCTION
  33. 33. DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTS
  34. 34. DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE UI TESTS public function testJobCannotBeFound() {
 $jobRepository = $this->prophesize(JobRepository::class);
 $jobRepository->getById(EXPECTED_JOB_ID)
 ->shouldBeCalled()
 ->willReturn(false);
 
 $jobService = new JobService($jobRepository->reveal());
 
 $this->assertFalse($jobService->getById(EXPECTED_JOB_ID));
 }
  35. 35. DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTS public function testFindJob() {
 $expectedJob = $this->loadFixture('active_job.yml');
 $actualJob = $this->repository->getById($expectedJob->getId());
 
 self::assertInstanceOf(Job::class, $actualJob);
 self::assertEquals($expectedJob->getId(), $actualJob->getId());
 }
  36. 36. DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTSScenario: Link to related job
 Given a job exists
 And there are related jobs available
 When that job is viewed
 Then a list of related jobs is shown
 And each related job links to the detail page of the related job
  37. 37. DEFENSE IN DEPTH UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI TESTS
  38. 38. DEFENSE IN DEPTH MANUAL TESTING? UNIT TESTS INTEGRATION
 TESTS ACCEPTANCE
 TESTS UI
  39. 39. CONTINUOUS TESTING UNIT TESTS INTEGRATION TESTS ACCEPTANCE TESTS UI TESTS SMOKE
 TESTS Cost Speed Exploratory
 testing Monitoring
  40. 40. PIPELINE AS CODE node {
 stage 'Run tests'
 sh "phpunit"
 sh "behat"
 
 stage 'Build docker image'
 sh "phing build"
 sh "docker build -t jobservice:${env.BUILD_NUMBER} ."
 sh "docker push jobservice:${env.BUILD_NUMBER}"
 
 stage 'Deploy acceptance'
 sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER}
 -i acc deploy.yml"
 
 stage 'Deploy production'
 sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER}
 -i prod deploy.yml"
 }
  41. 41. DOCKERFILE FROM php:7.0-apache ADD vhost.conf /etc/apache2/sites-available/000-default.conf ADD . /var/www/html
  42. 42. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER
  43. 43. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER docker pull
  44. 44. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER docker run
  45. 45. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER wait_for: port=8080 delay=5 timeout=15
  46. 46. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER uri:
 url: http://localhost:8080/_health
 status_code: 200
 timeout: 30
  47. 47. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER template: src=haproxy.cfg.j2
 dest=/etc/haproxy/haproxy.cfg service: name=haproxy state=reloaded
  48. 48. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER template: src=haproxy.cfg.j2
 dest=/etc/haproxy/haproxy.cfg service: name=haproxy state=reloaded
  49. 49. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER docker stop docker rm
  50. 50. BUILD PIPELINE
  51. 51. FEEDBACK!
  52. 52. RESULTS
  53. 53. RESULTS ▸ Total build time per service < 10 minutes ▸ Significantly improved page load times ▸ Improved audience stats (time on page, pages per session, session duration, traffic, seo ranking, etc) ▸ Increased confidence and velocity ▸ Experimented with new tech/stacks (angular, jvm, event sourcing) ▸ More fun
  54. 54. LESSONS LEARNED ▸ Team acceptance ▸ Change is hard ▸ Overhead of weekly sprint; requires discipline ▸ Docker orchestration ▸ Issues with traffic between Amazon <-> on-premise datacenter ▸ Javascript testing
  55. 55. LESSONS LEARNED ▸ Experience with new tech ▸ Stability of build pipelines ▸ Management/leadership buy-in ▸ Business alignment ▸ Not enough focus on replacing legacy application
  56. 56. QUESTION TIME
  57. 57. THANK YOU! @michieltcs / michiel@make.io

×