Remediation patterns
Upcoming SlideShare
Loading in...5
×
 

Remediation patterns

on

  • 3,601 views

There's a recording of this presentation from QCon London 2011: http://www.infoq.com/presentations/Remediation-Patterns

There's a recording of this presentation from QCon London 2011: http://www.infoq.com/presentations/Remediation-Patterns

Statistics

Views

Total Views
3,601
Views on SlideShare
3,522
Embed Views
79

Actions

Likes
2
Downloads
81
Comments
0

6 Embeds 79

http://www.scoop.it 47
http://www.linkedin.com 22
https://www.linkedin.com 6
http://paper.li 2
http://twitter.com 1
http://www.mefeedia.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Remediation patterns Remediation patterns Presentation Transcript

  • Remediation Patterns October 12, 2011, GOTO Amsterdam Jez Humble, ThoughtWorks Studios @jezhumble #continuousdelivery http://continuousdelivery.com/ © 2012 ThoughtWorks, Inc.
  • remediationITIL: “Recovery to a known state a er afailed Change or Release.”Recovery: “Returning a Con guration Item oran IT Service to a working state.”Jez: “Fixing shit when it breaks”
  • strategies for remediationpreventionpatterns for low-risk releasepatterns for incremental delivery
  • 1oz of prevention Diagram invented by Brian Marick
  • deployment pipelineDelivery team Version control Build & unit Automated User acceptance Release tests acceptance tests tests Check in Trigger Feedback Check in Trigger Feedback Trigger Feedback Check in Trigger Feedback Trigger Feedback Approval Feedback Approval
  • the hard bitstesting on production environmentscreating maintainable acceptance teststesting cross-functional requirements
  • reducing release riskoptimize for resilienceautomate provisioning and deploymentdevs, testers and ops collaborate throughout
  • optimize for TTRhttp://www.flickr.com/photos/chuecy/2629242132/ MTBF MTTR Thanks to John Allspaw: http://slideshare.net/jallspaw/
  • infrastructure as code Destroy works of art
  • low risk releases are incremental DEPENDENT SERVICE 1.0 1.1 Database Abstraction layer Abstraction layer STATIC CONTENT /static/1.1 APPLICATION /static/1.0 Router / Load balancer Interwebs
  • canary releasing Diagram by Martin Fowler
  • canary releasing Diagram by Martin Fowler
  • canary releasingreduce risk of releaseA/B testingperformance testing
  • immune systemwhat if someone replaced your“buy” button with spacer.gif? T cells http://www.flickr.com/photos/gehealthcare/3326186490/
  • monitoringBusiness metrics - revenue, # orders, # usersOps metrics - changes, incidents, TTD, TTR,TBFTechnical metrics - TPS, response time, hits http://www.flickr.com/photos/wwarby/3296379139/
  • the hard bitsanalyzing root causescollaborationdata
  • incremental delivery John Allspaw: “Ops Metametrics” http://slidesha.re/dsSZIr
  • incremental deliverydevelop on mainlinefeature toggles and branch by abstractiondark launching
  • feature toggles Config File[featureToggles] some.jspwobblyFoobars: trueflightyForkHandles: false <toggle name=wobblyFoobars> ... various UI elements </toggle> other.java forkHandle = (featureConfig.isOn(‘flightlyForkHandles)) ? new FlightyForkHander(aCandle) : new ForkHandler(aCandle) Stolen from Martin Fowler http://martinfowler.com/bliki/FeatureToggle.html
  • branch by abstraction Component A A Component Seam Component B
  • branch by abstraction Component A A Component Abstraction layerComponent B Component B’
  • dark launching Diagram by Martin Fowler
  • dark launching Diagram by Martin Fowler
  • measuring effectivenessHow long would it take you to release achange to a single line of code?Ops metrics - changes, incidents, TTD, TTR,TBFIf your data center blew up, how long wouldyou take to restore service?
  • questionsjez@thoughtworks.comhttp://continuousdelivery.com/@jezhumble #continuousdelivery © 2012 ThoughtWorks, Inc. http://thoughtworks-studios.com/