Increasing Agility
by Understanding Risk
Simon Croome
simon@croome.org
$ whoami
Simon Croome
Infrastructure Engineer
Financial Industry
Independent Consultant
Why Puppet?
Velocity
Configuration
Management is only part
of the solution
Software is deployable
throughout its lifecycle
Software is deployable
throughout its lifecycle
!
Priority is keeping the
software deployable over
new features
Software is deployable
throughout its lifecycle
!
Priority is keeping the
software deployable over
new features
!
Anybody ...
Software is deployable
throughout its lifecycle
!
Priority is keeping the
software deployable over
new features
!
Anybody ...
Continuous Delivery
Continuous Delivery
Reduces the transaction cost of making change
Continuous Delivery
Reduces the transaction cost of making change
Faster ROI
Continuous Delivery
Reduces the transaction cost of making change
Faster ROI
Reduces Risk
Risk
Concept: John Allspaw
http://blog.vincentbrouillet.com/costs-and-risks-benefits-of-
continuous-delivery-in-one-picture/
!
If it hurts
Do it more often
Controls
Controls
Change Management
Controls
Change Management
Separation of Duties
Controls
Change Management
Separation of Duties
Audit
An example workflow
Jira Ticket Engineering
Puppet
Data
Code
Branch
Puppet
Modules
Test
Servers
Dev
Puppet
Engineering
UNIX team member is ass...
Jira Ticket Engineering
Puppet
Data
Code
Branch
Puppet
Modules
Test
Servers
Dev
Puppet
Engineering
UNIX team member is ass...
Jira Ticket Engineering
Puppet
Data
Code
Branch
Puppet
Modules
Test
Servers
Dev
Puppet
Engineering
UNIX team member is ass...
Puppet
Data
Puppet
Modules
et.
sing Stash
Peer
Review
Code
Format
Tests
Integr’tn
Tests
Unit
Tests
Code Release
Once code ...
Peer
Review
Code Release
Once code is merged into Stash’s Release Branch, a UNIX team
member may create a release containi...
Initiate Deployment
Release is selected by
UNIX team member.
Select
Release
Dry-run
Change
Detail
Impacted
Roles
Build
Sta...
nitiate Deployment
Release is selected by
UNIX team member.
Select
Release
Dry-run
Change
Detail
Impacted
Roles
Build
Stat...
nge
ail
ld
tus
Manual
Review
read-only mode.
ack changes.
t and assigns risk.
Scheduling & Change Management
Each impacted...
Management
nters separate change management
hange risk feed into scheduling.
T
EBF
PERF
EBF
BCP
PROD
Deployment
Completed ...
DEV
SIT
DIT
EBF
PERF
EBF
Impacted
Roles
BCP
PROD
CAT
Approved
CHG Tickets
Risk
Maintenance
Windows ServiceNow
CHG
Tickets
...
Jira Ticket Engineering
Puppet
Data
Code
Branch
Puppet
Modules
Test
Servers
Dev
Puppet
Engineering
UNIX team member is ass...
Technology Stack
!
Puppet - Dynamic environments
MCollective - Comms
CouchDB - Reports
PouchDB - Replication to browser
An...
github.com/croomes/gonzo
!
Questions?
Gonzo - Increasing Agility by Understanding Risk
Gonzo - Increasing Agility by Understanding Risk
Gonzo - Increasing Agility by Understanding Risk
Upcoming SlideShare
Loading in …5
×

Gonzo - Increasing Agility by Understanding Risk

184 views
126 views

Published on

Gonzo lets you assess the impact of your Puppet change before rolling it out across your server estate. It takes over after continuous integration tools have run their tests and deployed the release to the Puppet Masters, but before clients have been updated.

Gonzo's goal is to increase confidence in Puppet changes by making it easier to verify that all changes are intentional and understood.

github.com/croomes/gonzo

Published in: Software, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
184
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Gonzo - Increasing Agility by Understanding Risk

  1. 1. Increasing Agility by Understanding Risk Simon Croome simon@croome.org
  2. 2. $ whoami Simon Croome Infrastructure Engineer Financial Industry Independent Consultant
  3. 3. Why Puppet?
  4. 4. Velocity
  5. 5. Configuration Management is only part of the solution
  6. 6. Software is deployable throughout its lifecycle
  7. 7. Software is deployable throughout its lifecycle ! Priority is keeping the software deployable over new features
  8. 8. Software is deployable throughout its lifecycle ! Priority is keeping the software deployable over new features ! Anybody can get fast, automated feedback on the production readiness of their systems any time a change is made
  9. 9. Software is deployable throughout its lifecycle ! Priority is keeping the software deployable over new features ! Anybody can get fast, automated feedback on the production readiness of their systems any time a change is made ! Push-button deployments of any version to any environment
  10. 10. Continuous Delivery
  11. 11. Continuous Delivery Reduces the transaction cost of making change
  12. 12. Continuous Delivery Reduces the transaction cost of making change Faster ROI
  13. 13. Continuous Delivery Reduces the transaction cost of making change Faster ROI Reduces Risk
  14. 14. Risk
  15. 15. Concept: John Allspaw http://blog.vincentbrouillet.com/costs-and-risks-benefits-of- continuous-delivery-in-one-picture/ !
  16. 16. If it hurts Do it more often
  17. 17. Controls
  18. 18. Controls Change Management
  19. 19. Controls Change Management Separation of Duties
  20. 20. Controls Change Management Separation of Duties Audit
  21. 21. An example workflow
  22. 22. Jira Ticket Engineering Puppet Data Code Branch Puppet Modules Test Servers Dev Puppet Engineering UNIX team member is assigned ticket. Jira creates development sandbox using Stash integration. Peer Review Code Format Tests Integr’tn Tests Unit Tests Code Release Once code is merged into Stash’s Release Branch, a UNIX team member may create a release containing one or more changes. Bamboo automates deployment of the release to UAT, further automated testing, then release to Production. Note that it will not yet be deployed to client systems. Create Release Deploy to UAT Integration Tests Deploy to PROD New Requirement Ticket is raised against UNIX Jira Issue tracker, e.g. New project, change to OS build. Test failures block the release. Initiate Deployment Release is selected by UNIX team member. Select Release Dry-run Change Detail Impacted Roles Build Status Non- Impacted Servers Impacted Servers Manual Review Impact Analysis Change is run across the environment in read-only mode. Servers that would be changed report back changes. UNIX Change Manager assesses impact and assigns risk. Scheduling & Change Management Each impacted server role enters separate change management workflow Maintenance windows and change risk feed into scheduling. DEV SIT DIT EBF PERF EBF Impacted Roles BCP PROD CAT Deployment Completed approvals trigger deployment during change window. . Approved CHG Tickets Apply Change Review Request Risk Maintenance Windows ServiceNow CHG Tickets Reporting Web console to track deployment of changes across the environment, including summary view of health by server role and business area. Un-approved tickets reset change workflow. Code Review UNIX team member creates “Pull Request”, indicating change is ready to be reviewed. Bamboo automates code quality review, unit and integration testing. If all tests pass, Engineering team alerted to perform peer review in Stash.
  23. 23. Jira Ticket Engineering Puppet Data Code Branch Puppet Modules Test Servers Dev Puppet Engineering UNIX team member is assigned ticket. Jira creates development sandbox using Stash integration. New Requirement Ticket is raised against UNIX Jira Issue tracker, e.g. New project, change to OS build. Test fail
  24. 24. Jira Ticket Engineering Puppet Data Code Branch Puppet Modules Test Servers Dev Puppet Engineering UNIX team member is assigned ticket. Jira creates development sandbox using Stash integration. New Requirement Ticket is raised against UNIX Jira Issue tracker, e.g. New project, change to OS build. Test failures block the release. Review Request Code Review UNIX team member change is ready to b Bamboo automates integration testing. If all tests pass, Eng peer review in Stash
  25. 25. Puppet Data Puppet Modules et. sing Stash Peer Review Code Format Tests Integr’tn Tests Unit Tests Code Release Once code is merged into member may create a rel Bamboo automates deplo automated testing, then r Note that it will not yet be Create Release Deploy UAT Test failures block the release. Review Request Code Review UNIX team member creates “Pull Request”, indicating change is ready to be reviewed. Bamboo automates code quality review, unit and integration testing. If all tests pass, Engineering team alerted to perform peer review in Stash.
  26. 26. Peer Review Code Release Once code is merged into Stash’s Release Branch, a UNIX team member may create a release containing one or more changes. Bamboo automates deployment of the release to UAT, further automated testing, then release to Production. Note that it will not yet be deployed to client systems. Create Release Deploy to UAT Integration Tests Deploy to PROD st”, indicating unit and to perform
  27. 27. Initiate Deployment Release is selected by UNIX team member. Select Release Dry-run Change Detail Impacted Roles Build Status Non- Impacted Servers Impacted Servers Manual Review Impact Analysis Change is run across the environment in read-only mode. Servers that would be changed report back changes. UNIX Change Manager assesses impact and assigns risk.
  28. 28. nitiate Deployment Release is selected by UNIX team member. Select Release Dry-run Change Detail Impacted Roles Build Status Non- Impacted Servers Impacted Servers Manual Review Impact Analysis Change is run across the environment in read-only mode. Servers that would be changed report back changes. UNIX Change Manager assesses impact and assigns risk. Scheduling & Ch Each impacted serv workflow Maintenance window DE Impacted Roles Reporting
  29. 29. nge ail ld tus Manual Review read-only mode. ack changes. t and assigns risk. Scheduling & Change Management Each impacted server role enters separate change management workflow Maintenance windows and change risk feed into scheduling. DEV SIT DIT EBF PERF EBF Impacted Roles BCP PROD CAT Deployment Completed app window. . Approved CHG Ticke Risk Maintenance Windows ServiceNow CHG Tickets Reporting Un-approved tickets r change workflow.
  30. 30. Management nters separate change management hange risk feed into scheduling. T EBF PERF EBF BCP PROD Deployment Completed approvals trigger deployment during change window. . Approved CHG Tickets Apply Change Risk Maintenance Windows ServiceNow CHG Tickets Un-approved tickets reset change workflow.
  31. 31. DEV SIT DIT EBF PERF EBF Impacted Roles BCP PROD CAT Approved CHG Tickets Risk Maintenance Windows ServiceNow CHG Tickets Reporting Web console to track deployment of changes across the environment, including summary view of health by server role and business area. Un-approved tickets reset change workflow.
  32. 32. Jira Ticket Engineering Puppet Data Code Branch Puppet Modules Test Servers Dev Puppet Engineering UNIX team member is assigned ticket. Jira creates development sandbox using Stash integration. Peer Review Code Format Tests Integr’tn Tests Unit Tests Code Release Once code is merged into Stash’s Release Branch, a UNIX team member may create a release containing one or more changes. Bamboo automates deployment of the release to UAT, further automated testing, then release to Production. Note that it will not yet be deployed to client systems. Create Release Deploy to UAT Integration Tests Deploy to PROD New Requirement Ticket is raised against UNIX Jira Issue tracker, e.g. New project, change to OS build. Test failures block the release. Initiate Deployment Release is selected by UNIX team member. Select Release Dry-run Change Detail Impacted Roles Build Status Non- Impacted Servers Impacted Servers Manual Review Impact Analysis Change is run across the environment in read-only mode. Servers that would be changed report back changes. UNIX Change Manager assesses impact and assigns risk. Scheduling & Change Management Each impacted server role enters separate change management workflow Maintenance windows and change risk feed into scheduling. DEV SIT DIT EBF PERF EBF Impacted Roles BCP PROD CAT Deployment Completed approvals trigger deployment during change window. . Approved CHG Tickets Apply Change Review Request Risk Maintenance Windows ServiceNow CHG Tickets Reporting Web console to track deployment of changes across the environment, including summary view of health by server role and business area. Un-approved tickets reset change workflow. Code Review UNIX team member creates “Pull Request”, indicating change is ready to be reviewed. Bamboo automates code quality review, unit and integration testing. If all tests pass, Engineering team alerted to perform peer review in Stash.
  33. 33. Technology Stack ! Puppet - Dynamic environments MCollective - Comms CouchDB - Reports PouchDB - Replication to browser AngularJS - Web interface Rails - API (to be replaced) Resque / Redis - Job scheduling !
  34. 34. github.com/croomes/gonzo ! Questions?

×