Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Cyansible Blending Blue/Green Deployments @Betterment
Techcrunch Disrupt May 2010 90,000 customers, more every minute. Fastest growing automated investing service $2B+ Who are ...
betterment : investing :: devops : engineering
it’s 2012. let’s ship some code.
Betterment@2012
A Better Migration: From Snowflakes to Stormtroopers Wednesday, July 22, 6:30p - 8:00p @ AWS Pop-up Loft |Shameless Plug:
it’s 2015. let’s ship new code… without interrupting production
● Predictable ● Repeatable ● Minimal Human Interaction ● Zero User Interruption ● Contained Failure Dream Delivery
Blue/Green Deployments http://martinfowler.com/bliki/BlueGreenDeployment.html
DNSELB
Pre-flight Checklist ❏Stateless Servers ❏Server Health Check ❏Duplicate Full Stack including RDS Replica
Wait. Two Databases? “There's still the issue of dealing with missed transactions while the green environment was live, bu...
“Code𝓲 always works on Schema𝓲+1” (A.K.A. old code works on new schema)
Publish, Migrate, Deploy
Jenkins’ Job 1. Build 2. Test 3. Package 4. Publish 5. Run Migrations 6. Invoke Ansible 7. Cull Zombies
Ansible’s Job 1. Check for S3 deliverables 2. Spin up new EC2 Instance(s) 3. Apply role(s) to instance(s) 4. Find instance...
Bootstrapping Ansible ./exec/ directory contains Jenkins entry points Ansible code lives in the repo it provisions. Publis...
./exec/brochure-deploy.sh
brochure-deploy.yml you are here. 00:00:10.888
provision-new-ec2- instances brochure-deploy >
identify-elb- AZs brochure-deploy > provision-new-ec2-instances >
ec2- deploy brochure-deploy > provision-new-ec2-instances >
AMI Hierarchy
brochure-deploy.yml you are here. 00:01:18.415
configure-instances brochure-deploy >
roles/deploy/brochure/tasks/main.y ml brochure-deploy > configure-instances
brochure-deploy.yml you are here. 00:04:49.414
brochure-deploy.yml you are here. 00:04:50.188
find-instances-in-elb brochure-deploy >
brochure-deploy.yml you are here. 00:04:50.911
add-instances-to-elb brochure-deploy >
brochure-deploy.yml you are here. 00:05:06.295
decommission-instances brochure-deploy >
brochure-deploy.yml you are here. 00:05:21.226
EC2 INSTANCE
● Predictable ● Repeatable ● Minimal Human Interaction ● Zero User Interruption ● Contained Failure Dream Delivery Achieved
The Future ● Long Running Instances + Docker o Huge speed improvement ● Post Monolith, Abandon Jenkins? o Travis CI for Bu...
Questions? alan@betterment.com @nonrational github.com/nonrational
careers@betterment.com All code snippets & diagrams contained in this presentation are property of Betterment, but please ...
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Cyansible
Upcoming SlideShare
Loading in …5
×

Cyansible

8,917 views

Published on

Blue/Green deployments have been an important, if rarely implemented, technique in the Continuous Delivery playbook for years. Their aim is simple: provision, deploy, test — and optionally roll-back — your application before it's served to the public. Betterment's deployment architecture takes a similar, but more straightforward approach, accomplishing the important goals sought out by Blue/Green practitioners. Dubbed 'Cyan' (a mixture of Blue/Green), Betterment uses Ansible to provision new instances, push the latest artifacts to them, and ensure that they're healthy before marking them ready for production. All this ensures fast, stable, zero-downtime rollout with minimal human interaction. We'll discuss Betterment's philosophical approach to shipping new code and then dive into the nitty-gritty Ansible that powers the whole thing.

Published in: Software
no profile picture user

  • Be the first to comment

Show More

Cyansible

  1. 1. Cyansible Blending Blue/Green Deployments @Betterment
  2. 2. Techcrunch Disrupt May 2010 90,000 customers, more every minute. Fastest growing automated investing service $2B+ Who are we?
  3. 3. betterment : investing :: devops : engineering
  4. 4. it’s 2012. let’s ship some code.
  5. 5. Betterment@2012
  6. 6. A Better Migration: From Snowflakes to Stormtroopers Wednesday, July 22, 6:30p - 8:00p @ AWS Pop-up Loft |Shameless Plug:
  7. 7. it’s 2015. let’s ship new code… without interrupting production
  8. 8. ● Predictable ● Repeatable ● Minimal Human Interaction ● Zero User Interruption ● Contained Failure Dream Delivery
  9. 9. Blue/Green Deployments http://martinfowler.com/bliki/BlueGreenDeployment.html
  10. 10. DNSELB
  11. 11. Pre-flight Checklist ❏Stateless Servers ❏Server Health Check ❏Duplicate Full Stack including RDS Replica
  12. 12. Wait. Two Databases? “There's still the issue of dealing with missed transactions while the green environment was live, but depending on your design you may be able to... ● feed transactions to both environments in such a way as to keep the blue environment as a backup when the green is live. Or you may be able to... ● put the application in read-only mode before cut-over, run it for a while in read-only mode, and then switch it to read-write mode.” http://martinfowler.com/bliki/BlueGreenDeployment.html
  13. 13. “Code𝓲 always works on Schema𝓲+1” (A.K.A. old code works on new schema)
  14. 14. Publish, Migrate, Deploy
  15. 15. Jenkins’ Job 1. Build 2. Test 3. Package 4. Publish 5. Run Migrations 6. Invoke Ansible 7. Cull Zombies
  16. 16. Ansible’s Job 1. Check for S3 deliverables 2. Spin up new EC2 Instance(s) 3. Apply role(s) to instance(s) 4. Find instance(s) in ELB 5. Add new instance(s) to ELB & tag o status: in-use 6. Remove & tag instances o status: zombie
  17. 17. Bootstrapping Ansible ./exec/ directory contains Jenkins entry points Ansible code lives in the repo it provisions. Publish it like an app.
  18. 18. ./exec/brochure-deploy.sh
  19. 19. brochure-deploy.yml you are here. 00:00:10.888
  20. 20. provision-new-ec2- instances brochure-deploy >
  21. 21. identify-elb- AZs brochure-deploy > provision-new-ec2-instances >
  22. 22. ec2- deploy brochure-deploy > provision-new-ec2-instances >
  23. 23. AMI Hierarchy
  24. 24. brochure-deploy.yml you are here. 00:01:18.415
  25. 25. configure-instances brochure-deploy >
  26. 26. roles/deploy/brochure/tasks/main.y ml brochure-deploy > configure-instances
  27. 27. brochure-deploy.yml you are here. 00:04:49.414
  28. 28. brochure-deploy.yml you are here. 00:04:50.188
  29. 29. find-instances-in-elb brochure-deploy >
  30. 30. brochure-deploy.yml you are here. 00:04:50.911
  31. 31. add-instances-to-elb brochure-deploy >
  32. 32. brochure-deploy.yml you are here. 00:05:06.295
  33. 33. decommission-instances brochure-deploy >
  34. 34. brochure-deploy.yml you are here. 00:05:21.226
  35. 35. EC2 INSTANCE
  36. 36. ● Predictable ● Repeatable ● Minimal Human Interaction ● Zero User Interruption ● Contained Failure Dream Delivery Achieved
  37. 37. The Future ● Long Running Instances + Docker o Huge speed improvement ● Post Monolith, Abandon Jenkins? o Travis CI for Build/Test o Tower for Deployment Orchestration ● Ansible Galaxy?
  38. 38. Questions? alan@betterment.com @nonrational github.com/nonrational
  39. 39. careers@betterment.com All code snippets & diagrams contained in this presentation are property of Betterment, but please learn from them. All photographs / GIFs used in this presentation are someone else’s. Street Fighter, Back To The Future, Indiana Jones, Futurama, and Arrested Development are someone else’s property too.

×